[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:34 dist-test-slave-39l8 krb5kdc[13953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:34 dist-test-slave-39l8 krb5kdc[13953](info): set up 2 sockets
Aug 08 11:43:34 dist-test-slave-39l8 krb5kdc[13953](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 11:43:35.869573 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:43:35 dist-test-slave-39l8 krb5kdc[13953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:43:36.055603 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:35387
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:35387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:36.386624 13979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:36.387017 13979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:36.387249 13979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:36.394344 13979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:36.394444 13979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:36.394532 13979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:36.394646 13979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:36.394723 13979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:36.404356 13979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:35387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:35387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:36.405673 13979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:36.407127 13979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:36.419158 13990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:36.419250 13987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:36.419212 13988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:36.419883 13979 server_base.cc:1034] running on GCE node
I20260808 11:43:36.420667 13979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:36.422013 13979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:36.423230 13979 hybrid_clock.cc:648] HybridClock initialized: now 1786189416423217 us; error 62 us; skew 500 ppm
I20260808 11:43:36.423540 13979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:36 dist-test-slave-39l8 krb5kdc[13953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189416, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:36.426501 13979 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:43:36.427964 13979 webserver.cc:460] Webserver started at http://127.13.158.126:36485/ using document root <none> and password file <none>
I20260808 11:43:36.428579 13979 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:36.428678 13979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:36.428915 13979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:36.431869 13979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "4a54c5279dcf49979bb0c525291235d2"
format_stamp: "Formatted at 2026-08-08 11:43:36 on dist-test-slave-39l8"
server_key: "d18ac08cce579dbdafd242b708c0a932"
server_key_iv: "4e3174ca069d482be1759db28ebf61d0"
server_key_version: "encryptionkey@0"
I20260808 11:43:36.432581 13979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "4a54c5279dcf49979bb0c525291235d2"
format_stamp: "Formatted at 2026-08-08 11:43:36 on dist-test-slave-39l8"
server_key: "d18ac08cce579dbdafd242b708c0a932"
server_key_iv: "4e3174ca069d482be1759db28ebf61d0"
server_key_version: "encryptionkey@0"
I20260808 11:43:36.437680 13979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 11:43:36.441051 14002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:36.442344 13979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:43:36.442523 13979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "4a54c5279dcf49979bb0c525291235d2"
format_stamp: "Formatted at 2026-08-08 11:43:36 on dist-test-slave-39l8"
server_key: "d18ac08cce579dbdafd242b708c0a932"
server_key_iv: "4e3174ca069d482be1759db28ebf61d0"
server_key_version: "encryptionkey@0"
I20260808 11:43:36.442744 13979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:36.468295 13979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:36.472635 13979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:36.473079 13979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:36.492628 13979 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:35387
I20260808 11:43:36.492653 14081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:35387 every 8 connection(s)
I20260808 11:43:36.494212 13979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:36.497984 14082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:36.504866 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 13979
I20260808 11:43:36.505169 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:43:36.507025 13945 external_mini_cluster.cc:1428] Setting key fba0eaa6e47db79785f8689d22ea8318
I20260808 11:43:36.508035 14082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2: Bootstrap starting.
I20260808 11:43:36.511512 14082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:36.512881 14082 log.cc:824] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:36.516144 14082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2: No bootstrap required, opened a new log
I20260808 11:43:36.527086 14082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54c5279dcf49979bb0c525291235d2" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35387 } }
I20260808 11:43:36.527588 14082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:36.527710 14082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a54c5279dcf49979bb0c525291235d2, State: Initialized, Role: FOLLOWER
I20260808 11:43:36.528431 14082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54c5279dcf49979bb0c525291235d2" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35387 } }
I20260808 11:43:36.528734 14082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:36.528906 14082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:36.529117 14082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:36.532219 14082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54c5279dcf49979bb0c525291235d2" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35387 } }
I20260808 11:43:36.532732 14082 leader_election.cc:302] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a54c5279dcf49979bb0c525291235d2; no voters:
I20260808 11:43:36.533458 14082 leader_election.cc:288] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:36.533558 14090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:36.534775 14090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [term 1 LEADER]: Becoming Leader. State: Replica: 4a54c5279dcf49979bb0c525291235d2, State: Running, Role: LEADER
I20260808 11:43:36.535346 14090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54c5279dcf49979bb0c525291235d2" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35387 } }
I20260808 11:43:36.535868 14082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:36.538877 14092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a54c5279dcf49979bb0c525291235d2. Latest consensus state: current_term: 1 leader_uuid: "4a54c5279dcf49979bb0c525291235d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54c5279dcf49979bb0c525291235d2" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35387 } } }
I20260808 11:43:36.538908 14091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a54c5279dcf49979bb0c525291235d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54c5279dcf49979bb0c525291235d2" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35387 } } }
I20260808 11:43:36.539261 14092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:36.539310 14091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2 [sys.catalog]: This master's current role is: LEADER
Aug 08 11:43:36 dist-test-slave-39l8 krb5kdc[13953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:36.547792 14101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:36.554150 14101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:36.559598 14086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:36.510754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.13.158.126:35387)
0808 11:43:36.511752 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:36.511769 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:36.513250 (+ 1481us) server_negotiation.cc:407] Connection header received
0808 11:43:36.526538 (+ 13288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:36.526560 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:36.526978 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:36.527283 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:36.531464 (+ 4181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:36.532888 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:36.535121 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:36.535996 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:36.536183 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:36.544973 (+ 8790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:36.544989 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:36.545013 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:36.545078 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:36.550423 (+ 5345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:36.551747 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:36.551760 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:36.551770 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:36.552062 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:36.552796 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:36.552804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:36.552810 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:36.553687 (+ 877us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:36.554138 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:36.557214 (+ 3076us) server_negotiation.cc:299] Negotiation successful
0808 11:43:36.557759 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":413,"threads_started":1}
I20260808 11:43:36.564576 14101 catalog_manager.cc:1347] Generated new cluster ID: 0a6b9e7015034a3a90ea823e6bf8215b
I20260808 11:43:36.564721 14101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:36.589339 14101 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:36.590615 14101 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:36.601222 14101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a54c5279dcf49979bb0c525291235d2: Generated new TSK 0
I20260808 11:43:36.602203 14101 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:43:36.662351 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35387
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:37.005436 14122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:37.005759 14122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:37.005867 14122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:37.013284 14122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:37.013406 14122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:37.013522 14122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:43:37.023587 14122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35387
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:37.024983 14122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:37.026314 14122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:37.040278 14131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:37.040335 14136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:37.040381 14134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:37.046412 14122 server_base.cc:1034] running on GCE node
I20260808 11:43:37.047049 14122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:37.048058 14122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:37.049275 14122 hybrid_clock.cc:648] HybridClock initialized: now 1786189417049223 us; error 36 us; skew 500 ppm
I20260808 11:43:37.049582 14122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:37 dist-test-slave-39l8 krb5kdc[13953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:37.052825 14122 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:43:37.054214 14122 webserver.cc:460] Webserver started at http://127.13.158.65:45629/ using document root <none> and password file <none>
I20260808 11:43:37.054996 14122 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:37.055099 14122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:37.055366 14122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:37.058593 14122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "6f3c9ab8f555417aa130313dbb7681b3"
format_stamp: "Formatted at 2026-08-08 11:43:37 on dist-test-slave-39l8"
server_key: "2e9c7b5c3580c6d73792012da0a6c494"
server_key_iv: "a0825fcd1a19a292b222080094d95f19"
server_key_version: "encryptionkey@0"
I20260808 11:43:37.059417 14122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "6f3c9ab8f555417aa130313dbb7681b3"
format_stamp: "Formatted at 2026-08-08 11:43:37 on dist-test-slave-39l8"
server_key: "2e9c7b5c3580c6d73792012da0a6c494"
server_key_iv: "a0825fcd1a19a292b222080094d95f19"
server_key_version: "encryptionkey@0"
I20260808 11:43:37.064669 14122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:43:37.068182 14143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:37.069471 14122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:43:37.069607 14122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "6f3c9ab8f555417aa130313dbb7681b3"
format_stamp: "Formatted at 2026-08-08 11:43:37 on dist-test-slave-39l8"
server_key: "2e9c7b5c3580c6d73792012da0a6c494"
server_key_iv: "a0825fcd1a19a292b222080094d95f19"
server_key_version: "encryptionkey@0"
I20260808 11:43:37.069774 14122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:37.100137 14122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:37.104898 14122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:37.105166 14122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:37.106118 14122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:43:37.107530 14122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:37.107609 14122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:37.107702 14122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:37.107755 14122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:37.143996 14122 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:33035
I20260808 11:43:37.144042 14293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:33035 every 8 connection(s)
I20260808 11:43:37.145378 14122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:43:37.153982 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14122
I20260808 11:43:37.154217 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:43:37.154896 13945 external_mini_cluster.cc:1428] Setting key 04b651761faaecfd1db82b078a8ceebe
Aug 08 11:43:37 dist-test-slave-39l8 krb5kdc[13953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:37.171044 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:37.148616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:35387 (local address 127.13.158.65:46379)
0808 11:43:37.149409 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:37.149460 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:37.151348 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:37.152160 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:37.152168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:37.152668 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:37.153213 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:37.153229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:37.154333 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:37.154342 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:37.155856 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:37.155871 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:37.156665 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:37.156674 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:37.156965 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:37.157674 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:37.157707 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:37.160600 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:37.165091 (+ 4491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:37.165126 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:37.165144 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:37.165895 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:37.166452 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:37.166460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:37.166463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:37.166787 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:37.167597 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:37.167609 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:37.168224 (+ 615us) client_negotiation.cc:769] Sending connection context
0808 11:43:37.168610 (+ 386us) client_negotiation.cc:240] Negotiation successful
0808 11:43:37.169316 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":325,"threads_started":1}
I20260808 11:43:37.172443 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:37.148173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:46379 (local address 127.13.158.126:35387)
0808 11:43:37.148717 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:37.148727 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:37.149596 (+ 869us) server_negotiation.cc:407] Connection header received
0808 11:43:37.151665 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:37.151671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:37.151772 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:37.151897 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:37.153393 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:37.154156 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:37.155986 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:37.156420 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:37.156569 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:37.160852 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:37.160862 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:37.160869 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:37.160919 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:37.164896 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:37.166055 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:37.166063 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:37.166068 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:37.166237 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:37.166947 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:37.166953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:37.166958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:37.167421 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:37.167585 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:37.171694 (+ 4109us) server_negotiation.cc:299] Negotiation successful
0808 11:43:37.171972 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":278,"threads_started":1}
I20260808 11:43:37.174320 14294 heartbeater.cc:347] Connected to a master server at 127.13.158.126:35387
I20260808 11:43:37.174659 14294 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:37.175662 14294 heartbeater.cc:511] Master 127.13.158.126:35387 requested a full tablet report, sending...
I20260808 11:43:37.179113 14027 ts_manager.cc:194] Registered new tserver with Master: 6f3c9ab8f555417aa130313dbb7681b3 (127.13.158.65:33035)
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
I20260808 11:43:37.181164 14027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:46379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:43:37.214382 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35387
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:37.541230 14304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:37.541487 14304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:37.541584 14304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:37.548681 14304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:37.548796 14304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:37.548916 14304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:43:37.558434 14304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35387
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:37.559652 14304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:37.560921 14304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:37.573947 14310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:37.573993 14313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:37.574069 14317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:37.575593 14304 server_base.cc:1034] running on GCE node
I20260808 11:43:37.576222 14304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:37.577677 14304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:37.579022 14304 hybrid_clock.cc:648] HybridClock initialized: now 1786189417578947 us; error 51 us; skew 500 ppm
I20260808 11:43:37.579421 14304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:37 dist-test-slave-39l8 krb5kdc[13953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:37.582845 14304 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:43:37.584334 14304 webserver.cc:460] Webserver started at http://127.13.158.66:40839/ using document root <none> and password file <none>
I20260808 11:43:37.585114 14304 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:37.585263 14304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:37.585654 14304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:37.588950 14304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "f768a0dac5e044869e168860a41462e7"
format_stamp: "Formatted at 2026-08-08 11:43:37 on dist-test-slave-39l8"
server_key: "a27add4f91188cdb21d4f2efc12e94d6"
server_key_iv: "bed7c36c963ba2ea034bf1a97d62ec3c"
server_key_version: "encryptionkey@0"
I20260808 11:43:37.589736 14304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "f768a0dac5e044869e168860a41462e7"
format_stamp: "Formatted at 2026-08-08 11:43:37 on dist-test-slave-39l8"
server_key: "a27add4f91188cdb21d4f2efc12e94d6"
server_key_iv: "bed7c36c963ba2ea034bf1a97d62ec3c"
server_key_version: "encryptionkey@0"
I20260808 11:43:37.594908 14304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:43:37.598280 14333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:37.599565 14304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:37.599740 14304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "f768a0dac5e044869e168860a41462e7"
format_stamp: "Formatted at 2026-08-08 11:43:37 on dist-test-slave-39l8"
server_key: "a27add4f91188cdb21d4f2efc12e94d6"
server_key_iv: "bed7c36c963ba2ea034bf1a97d62ec3c"
server_key_version: "encryptionkey@0"
I20260808 11:43:37.599928 14304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:37.624511 14304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:37.628520 14304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:37.628816 14304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:37.629783 14304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:43:37.631146 14304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:37.631251 14304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:37.631394 14304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:37.631469 14304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:37.666800 14304 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:36819
I20260808 11:43:37.666891 14508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:36819 every 8 connection(s)
I20260808 11:43:37.668104 14304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:43:37.673239 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14304
I20260808 11:43:37.673440 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:43:37.673946 13945 external_mini_cluster.cc:1428] Setting key 8850f765bb32a6f10bfed8c5eb04befc
Aug 08 11:43:37 dist-test-slave-39l8 krb5kdc[13953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:37.691147 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:37.670808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:51893 (local address 127.13.158.126:35387)
0808 11:43:37.670974 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:37.670979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:37.672258 (+ 1279us) server_negotiation.cc:407] Connection header received
0808 11:43:37.674367 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:37.674373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:37.674463 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:37.674571 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:37.676395 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:37.677044 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:37.679640 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:37.680073 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:37.680170 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:37.683929 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:37.683938 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:37.683945 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:37.683990 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:37.687669 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:37.688416 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:37.688424 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:37.688431 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:37.688608 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:37.689075 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:37.689082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:37.689086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:37.689473 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:37.689654 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:37.690359 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 11:43:37.690641 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:43:37.691715 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:37.671240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:35387 (local address 127.13.158.66:51893)
0808 11:43:37.672064 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:37.672142 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:37.674110 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:37.674884 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:37.674894 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:37.675427 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:37.676217 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:37.676242 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:37.677271 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:37.677279 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:37.679495 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:37.679513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:37.680299 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:37.680308 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:37.680613 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:37.681369 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:37.681401 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:37.683676 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:37.687828 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:37.687834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:37.687846 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:37.688271 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:37.688755 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:37.688759 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:37.688761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:37.688938 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:37.689647 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:37.689653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:37.689946 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 11:43:37.690186 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 11:43:37.690542 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":339,"threads_started":1}
I20260808 11:43:37.693256 14509 heartbeater.cc:347] Connected to a master server at 127.13.158.126:35387
I20260808 11:43:37.693552 14509 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
I20260808 11:43:37.694288 14509 heartbeater.cc:511] Master 127.13.158.126:35387 requested a full tablet report, sending...
I20260808 11:43:37.695736 14027 ts_manager.cc:194] Registered new tserver with Master: f768a0dac5e044869e168860a41462e7 (127.13.158.66:36819)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
I20260808 11:43:37.696578 14027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:51893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:43:37.718135 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35387
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:38.053397 14516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:38.053771 14516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:38.053942 14516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:38.060956 14516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:38.061107 14516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:38.061316 14516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:43:38.070919 14516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35387
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:38.072281 14516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:38.073648 14516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:38.086647 14523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:38.086725 14528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:38.086647 14522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:38.087553 14516 server_base.cc:1034] running on GCE node
I20260808 11:43:38.088122 14516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:38.089033 14516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:38.090267 14516 hybrid_clock.cc:648] HybridClock initialized: now 1786189418090226 us; error 48 us; skew 500 ppm
I20260808 11:43:38.090579 14516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[13953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189418, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:38.093631 14516 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:43:38.095170 14516 webserver.cc:460] Webserver started at http://127.13.158.67:40843/ using document root <none> and password file <none>
I20260808 11:43:38.095867 14516 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:38.096015 14516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:38.096340 14516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:38.099390 14516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "d8de9073c53a4b19a89486b73eac01f1"
format_stamp: "Formatted at 2026-08-08 11:43:38 on dist-test-slave-39l8"
server_key: "45a08b99ebe630f9f8277770338a8c33"
server_key_iv: "9995887450e98edec4859ab4df7fdfaf"
server_key_version: "encryptionkey@0"
I20260808 11:43:38.100150 14516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "d8de9073c53a4b19a89486b73eac01f1"
format_stamp: "Formatted at 2026-08-08 11:43:38 on dist-test-slave-39l8"
server_key: "45a08b99ebe630f9f8277770338a8c33"
server_key_iv: "9995887450e98edec4859ab4df7fdfaf"
server_key_version: "encryptionkey@0"
I20260808 11:43:38.105413 14516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:43:38.108908 14540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:38.110224 14516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:43:38.110404 14516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "d8de9073c53a4b19a89486b73eac01f1"
format_stamp: "Formatted at 2026-08-08 11:43:38 on dist-test-slave-39l8"
server_key: "45a08b99ebe630f9f8277770338a8c33"
server_key_iv: "9995887450e98edec4859ab4df7fdfaf"
server_key_version: "encryptionkey@0"
I20260808 11:43:38.110623 14516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:38.127548 14516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:38.131253 14516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:38.131537 14516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:38.132494 14516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:43:38.133841 14516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:38.133957 14516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:38.134107 14516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:38.134182 14516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:38.169377 14516 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:36133
I20260808 11:43:38.169395 14697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:36133 every 8 connection(s)
I20260808 11:43:38.170666 14516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:43:38.175623 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14516
I20260808 11:43:38.175822 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:43:38.176265 13945 external_mini_cluster.cc:1428] Setting key 6f8aa1b3c1cc1ad3d20d5d5a19a0a619
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[13953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189418, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:38.184445 14294 heartbeater.cc:503] Master 127.13.158.126:35387 was elected leader, sending a full tablet report...
I20260808 11:43:38.190876 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:38.173582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:51641 (local address 127.13.158.126:35387)
0808 11:43:38.173782 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:38.173790 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:38.174894 (+ 1104us) server_negotiation.cc:407] Connection header received
0808 11:43:38.176870 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:38.176876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:38.176979 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:38.177113 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:38.178840 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.179827 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.181560 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.181953 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.182052 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:38.185521 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:38.185526 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:38.185529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:38.185561 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:38.188263 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.188902 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.188906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.188908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.189024 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.189413 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.189416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.189418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.189668 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:38.189795 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:38.190358 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 11:43:38.190520 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:43:38.192116 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:38.173953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:35387 (local address 127.13.158.67:51641)
0808 11:43:38.174662 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:38.174753 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:38.176582 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:38.177413 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:38.177428 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:38.177959 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:38.178617 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:38.178644 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.180022 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.180026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:38.181421 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:38.181433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.182142 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.182147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:38.182336 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:38.183011 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:38.183037 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:38.185337 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:38.188385 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:38.188391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:38.188400 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:38.188791 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:38.189114 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:38.189117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:38.189119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:38.189310 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:38.189779 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:38.189785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:38.190051 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 11:43:38.190269 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 11:43:38.190605 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":324,"threads_started":1}
I20260808 11:43:38.194132 14698 heartbeater.cc:347] Connected to a master server at 127.13.158.126:35387
I20260808 11:43:38.194433 14698 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:38.195180 14698 heartbeater.cc:511] Master 127.13.158.126:35387 requested a full tablet report, sending...
I20260808 11:43:38.196544 14027 ts_manager.cc:194] Registered new tserver with Master: d8de9073c53a4b19a89486b73eac01f1 (127.13.158.67:36133)
I20260808 11:43:38.197327 14027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:51641
I20260808 11:43:38.205030 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[13953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189418, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[13953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189418, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:43:38.253443 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:38.237841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43550 (local address 127.13.158.65:33035)
0808 11:43:38.238528 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:38.238536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:38.238570 (+ 34us) server_negotiation.cc:407] Connection header received
0808 11:43:38.238649 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:38.238655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:38.238930 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:38.239094 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:38.240557 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.241554 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.242856 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.243463 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.243612 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:38.247538 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:38.247549 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:38.247562 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:38.247597 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:38.250479 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.251235 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.251242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.251245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.251363 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.251791 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.251801 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.251804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.252189 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:38.252510 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:38.252808 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 11:43:38.253069 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":337,"threads_started":1}
W20260808 11:43:38.254786 14178 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43550
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[13953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[13953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:43:38.300285 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:38.286437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.13.158.65:33035)
0808 11:43:38.286671 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:38.286678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:38.286958 (+ 280us) server_negotiation.cc:407] Connection header received
0808 11:43:38.287178 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:38.287188 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:38.287288 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:38.287405 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:38.288759 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.289454 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.290796 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.291269 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.291413 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:38.295315 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:38.295320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:38.295324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:38.295356 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:38.297677 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.298329 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.298336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.298339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.298465 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.298915 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.298921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.298926 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.299171 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:38.299442 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:38.299751 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 11:43:38.299970 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:43:38.301807 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14122
I20260808 11:43:38.313831 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:38.302823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.13.158.65:33035)
0808 11:43:38.302979 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:38.302983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:38.303430 (+ 447us) server_negotiation.cc:407] Connection header received
0808 11:43:38.303559 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:38.303562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:38.303614 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:38.303693 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:38.304766 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.305452 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.306615 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.307049 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.307139 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:38.309228 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:38.309233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:38.309236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:38.309264 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:38.311743 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.312334 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.312337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.312339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.312467 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.312770 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.312773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.312775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.313001 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:38.313172 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:38.313411 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 11:43:38.313552 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:43:38.314520 14289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:38.401240 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14122
I20260808 11:43:38.425976 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14304
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[13953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:43:38.441015 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:38.426910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43618 (local address 127.13.158.66:36819)
0808 11:43:38.427464 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:38.427471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:38.427497 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:43:38.427561 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:38.427568 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:38.427783 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:38.427914 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:38.429000 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.429952 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.431246 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.431781 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.431879 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:38.435776 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:38.435787 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:38.435800 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:38.435852 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:38.438565 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.439236 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.439243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.439245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.439390 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.439714 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.439717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.439719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.440054 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:38.440270 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:38.440534 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 11:43:38.440734 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":291,"threads_started":1}
I20260808 11:43:38.442217 14504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:38.530382 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14304
I20260808 11:43:38.553483 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14516
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[13953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:43:38.568928 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:38.554498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.13.158.67:36133)
0808 11:43:38.555191 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:38.555198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:38.555226 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:43:38.555287 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:38.555293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:38.555495 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:38.555643 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:38.556821 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.557871 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.559245 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:38.559788 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:38.559897 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:38.563792 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:38.563803 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:38.563815 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:38.563846 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:38.566594 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.567210 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.567216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.567219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.567363 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:38.567693 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:38.567696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:38.567698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:38.568015 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:38.568234 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:38.568449 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 11:43:38.568641 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":389,"threads_started":1}
I20260808 11:43:38.570158 14693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:38.651837 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14516
I20260808 11:43:38.673758 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13979
I20260808 11:43:38.675014 14071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:38.775848 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 13979
2026-08-08T11:43:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4272 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[14726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[14726](info): set up 2 sockets
Aug 08 11:43:38 dist-test-slave-39l8 krb5kdc[14726](info): commencing operation
krb5kdc: starting...
W20260808 11:43:40.028723 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:43:40 dist-test-slave-39l8 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:43:40.217988 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:46533
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:46533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:40.537384 14746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:40.537673 14746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:40.537864 14746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:40.544683 14746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:40.544795 14746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:40.544902 14746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:40.545001 14746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:40.545087 14746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:40.554265 14746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:46533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:46533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:40.555490 14746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:40.556783 14746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:40.568264 14760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:40.568336 14756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:40.568238 14757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:40.569090 14746 server_base.cc:1034] running on GCE node
I20260808 11:43:40.569890 14746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:40.571254 14746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:40.572460 14746 hybrid_clock.cc:648] HybridClock initialized: now 1786189420572413 us; error 40 us; skew 500 ppm
I20260808 11:43:40.572757 14746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:40 dist-test-slave-39l8 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:40.580075 14746 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:43:40.581451 14746 webserver.cc:460] Webserver started at http://127.13.158.126:42257/ using document root <none> and password file <none>
I20260808 11:43:40.582080 14746 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:40.582175 14746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:40.582449 14746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:40.585531 14746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "8b55d2885a6e4a8cb462582dd3f7a06b"
format_stamp: "Formatted at 2026-08-08 11:43:40 on dist-test-slave-39l8"
server_key: "0a251b680205d1b575babddc5bd328ff"
server_key_iv: "bdd6a128869cb0c6f51c67d0257e8ba0"
server_key_version: "encryptionkey@0"
I20260808 11:43:40.586194 14746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "8b55d2885a6e4a8cb462582dd3f7a06b"
format_stamp: "Formatted at 2026-08-08 11:43:40 on dist-test-slave-39l8"
server_key: "0a251b680205d1b575babddc5bd328ff"
server_key_iv: "bdd6a128869cb0c6f51c67d0257e8ba0"
server_key_version: "encryptionkey@0"
I20260808 11:43:40.592572 14746 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260808 11:43:40.595927 14769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:40.597203 14746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:43:40.597342 14746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "8b55d2885a6e4a8cb462582dd3f7a06b"
format_stamp: "Formatted at 2026-08-08 11:43:40 on dist-test-slave-39l8"
server_key: "0a251b680205d1b575babddc5bd328ff"
server_key_iv: "bdd6a128869cb0c6f51c67d0257e8ba0"
server_key_version: "encryptionkey@0"
I20260808 11:43:40.597508 14746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:40.630424 14746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:40.634806 14746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:40.635038 14746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:40.653270 14746 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:46533
I20260808 11:43:40.653312 14850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:46533 every 8 connection(s)
I20260808 11:43:40.654564 14746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:40.656389 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14746
I20260808 11:43:40.656648 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:43:40.657454 13945 external_mini_cluster.cc:1428] Setting key 200f3142282ffb9f5f9097f671f902d5
I20260808 11:43:40.658746 14851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:43:40 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:40.672776 14851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b: Bootstrap starting.
I20260808 11:43:40.677028 14851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:40.678341 14851 log.cc:824] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:40.681730 14851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b: No bootstrap required, opened a new log
I20260808 11:43:40.682107 14858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:40.659834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.13.158.126:46533)
0808 11:43:40.660681 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:40.660697 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:40.660754 (+ 57us) server_negotiation.cc:407] Connection header received
0808 11:43:40.662424 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:40.662450 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:40.662998 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:40.663434 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:40.664764 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:40.666752 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:40.667995 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:40.669037 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:40.669237 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:40.673535 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:40.673546 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:40.673564 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:40.673619 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:40.677732 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:40.678390 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:40.678398 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:40.678408 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:40.678607 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:40.679082 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:40.679088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:40.679098 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:40.679678 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:40.679966 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:40.680411 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 11:43:40.680820 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":363,"threads_started":1}
I20260808 11:43:40.689920 14851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b55d2885a6e4a8cb462582dd3f7a06b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46533 } }
I20260808 11:43:40.690237 14851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:40.690312 14851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b55d2885a6e4a8cb462582dd3f7a06b, State: Initialized, Role: FOLLOWER
I20260808 11:43:40.690874 14851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b55d2885a6e4a8cb462582dd3f7a06b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46533 } }
I20260808 11:43:40.691076 14851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:40.691172 14851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:40.691304 14851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:40.693328 14851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b55d2885a6e4a8cb462582dd3f7a06b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46533 } }
I20260808 11:43:40.693753 14851 leader_election.cc:302] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b55d2885a6e4a8cb462582dd3f7a06b; no voters:
I20260808 11:43:40.694272 14851 leader_election.cc:288] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:40.694434 14860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:40.695233 14860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [term 1 LEADER]: Becoming Leader. State: Replica: 8b55d2885a6e4a8cb462582dd3f7a06b, State: Running, Role: LEADER
I20260808 11:43:40.695772 14860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b55d2885a6e4a8cb462582dd3f7a06b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46533 } }
I20260808 11:43:40.696571 14851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:40.698756 14862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b55d2885a6e4a8cb462582dd3f7a06b. Latest consensus state: current_term: 1 leader_uuid: "8b55d2885a6e4a8cb462582dd3f7a06b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b55d2885a6e4a8cb462582dd3f7a06b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46533 } } }
I20260808 11:43:40.698729 14861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b55d2885a6e4a8cb462582dd3f7a06b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b55d2885a6e4a8cb462582dd3f7a06b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46533 } } }
I20260808 11:43:40.699190 14862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:40.699354 14861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:40.702179 14865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:40.707348 14865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:40.715854 14865 catalog_manager.cc:1347] Generated new cluster ID: 82c9d440f2c74e1c97acc5084ea667f1
I20260808 11:43:40.715986 14865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:40.730340 14865 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:40.732049 14865 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:40.744447 14865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b55d2885a6e4a8cb462582dd3f7a06b: Generated new TSK 0
I20260808 11:43:40.745434 14865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:43:40.794193 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46533
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:41.120822 14885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:41.121162 14885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:41.121338 14885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:41.128430 14885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:41.128580 14885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:41.128772 14885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:43:41.138543 14885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46533
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:41.139878 14885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:41.141224 14885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:41.154557 14897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:41.154649 14894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:41.154680 14895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:41.155686 14885 server_base.cc:1034] running on GCE node
I20260808 11:43:41.156234 14885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:41.157047 14885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:41.158246 14885 hybrid_clock.cc:648] HybridClock initialized: now 1786189421158169 us; error 60 us; skew 500 ppm
I20260808 11:43:41.158557 14885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:41 dist-test-slave-39l8 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189421, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:41.162000 14885 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:43:41.163595 14885 webserver.cc:460] Webserver started at http://127.13.158.65:34091/ using document root <none> and password file <none>
I20260808 11:43:41.164294 14885 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:41.164448 14885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:41.164773 14885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:41.167727 14885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "8c0a02f591e046268d7ea31b867d23d1"
format_stamp: "Formatted at 2026-08-08 11:43:41 on dist-test-slave-39l8"
server_key: "740ca00f4c7e288a1bbe498be2164dd0"
server_key_iv: "adb6e634bae43e6989178f6c6ef269fd"
server_key_version: "encryptionkey@0"
I20260808 11:43:41.168520 14885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "8c0a02f591e046268d7ea31b867d23d1"
format_stamp: "Formatted at 2026-08-08 11:43:41 on dist-test-slave-39l8"
server_key: "740ca00f4c7e288a1bbe498be2164dd0"
server_key_iv: "adb6e634bae43e6989178f6c6ef269fd"
server_key_version: "encryptionkey@0"
I20260808 11:43:41.173828 14885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:43:41.177361 14904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:41.178622 14885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:43:41.178870 14885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "8c0a02f591e046268d7ea31b867d23d1"
format_stamp: "Formatted at 2026-08-08 11:43:41 on dist-test-slave-39l8"
server_key: "740ca00f4c7e288a1bbe498be2164dd0"
server_key_iv: "adb6e634bae43e6989178f6c6ef269fd"
server_key_version: "encryptionkey@0"
I20260808 11:43:41.179086 14885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:41.191951 14885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:41.196307 14885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:41.196599 14885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:41.197656 14885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:43:41.199057 14885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:41.199129 14885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:41.199210 14885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:41.199258 14885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:41.232388 14885 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:42981
I20260808 11:43:41.232431 15082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:42981 every 8 connection(s)
I20260808 11:43:41.233830 14885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:43:41.234233 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14885
I20260808 11:43:41.234494 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:43:41.235360 13945 external_mini_cluster.cc:1428] Setting key 5e268a25665402a0319463a1c83c67fa
Aug 08 11:43:41 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189421, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
I20260808 11:43:41.256876 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:41.236677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:33495 (local address 127.13.158.126:46533)
0808 11:43:41.237159 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:41.237165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:41.238395 (+ 1230us) server_negotiation.cc:407] Connection header received
0808 11:43:41.240277 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:41.240285 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:41.240376 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:41.240500 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:41.241950 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:41.243086 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:41.245253 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:41.245632 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:41.245735 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:41.249960 (+ 4225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:41.249969 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:41.249976 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:41.250021 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:41.253707 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:41.254391 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:41.254398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:41.254402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:41.254552 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:41.254999 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:41.255004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:41.255008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:41.255385 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:41.255537 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:41.256222 (+ 685us) server_negotiation.cc:299] Negotiation successful
0808 11:43:41.256444 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":243,"threads_started":1}
I20260808 11:43:41.257548 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:41.237374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46533 (local address 127.13.158.65:33495)
0808 11:43:41.238216 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:41.238263 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:41.240013 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:41.240706 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:41.240714 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:41.241211 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:41.241773 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:41.241791 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:41.243276 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:41.243284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:41.245121 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:41.245134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:41.245858 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:41.245867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:41.246174 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:41.247004 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:41.247037 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:41.249686 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:41.253849 (+ 4163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:41.253856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:41.253867 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:41.254254 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:41.254658 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:41.254661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:41.254664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:41.254887 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:41.255506 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:41.255511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:41.255833 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 11:43:41.256037 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 11:43:41.256384 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":364,"threads_started":1}
I20260808 11:43:41.259138 15084 heartbeater.cc:347] Connected to a master server at 127.13.158.126:46533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
I20260808 11:43:41.259445 15084 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:41.260156 15084 heartbeater.cc:511] Master 127.13.158.126:46533 requested a full tablet report, sending...
I20260808 11:43:41.262393 14795 ts_manager.cc:194] Registered new tserver with Master: 8c0a02f591e046268d7ea31b867d23d1 (127.13.158.65:42981)
I20260808 11:43:41.264254 14795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:33495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:43:41.280009 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46533
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:41.609427 15091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:41.609746 15091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:41.609895 15091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:41.616858 15091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:41.617084 15091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:41.617295 15091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:43:41.626824 15091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46533
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:41.628058 15091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:41.629343 15091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:41.642338 15103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:41.642388 15104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:41.642771 15106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:41.643201 15091 server_base.cc:1034] running on GCE node
I20260808 11:43:41.643777 15091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:41.644671 15091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:41.645871 15091 hybrid_clock.cc:648] HybridClock initialized: now 1786189421645836 us; error 34 us; skew 500 ppm
I20260808 11:43:41.646202 15091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:41 dist-test-slave-39l8 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189421, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:41.649307 15091 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:43:41.650768 15091 webserver.cc:460] Webserver started at http://127.13.158.66:34617/ using document root <none> and password file <none>
I20260808 11:43:41.651420 15091 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:41.651547 15091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:41.651860 15091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:41.655565 15091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "40a4431d4f604537bcd77b969acef3da"
format_stamp: "Formatted at 2026-08-08 11:43:41 on dist-test-slave-39l8"
server_key: "0b2e2df6f3e9fbfb3ed61773e370559d"
server_key_iv: "6617f54eb48104af58ce0a0700c62b70"
server_key_version: "encryptionkey@0"
I20260808 11:43:41.656507 15091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "40a4431d4f604537bcd77b969acef3da"
format_stamp: "Formatted at 2026-08-08 11:43:41 on dist-test-slave-39l8"
server_key: "0b2e2df6f3e9fbfb3ed61773e370559d"
server_key_iv: "6617f54eb48104af58ce0a0700c62b70"
server_key_version: "encryptionkey@0"
I20260808 11:43:41.662545 15091 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260808 11:43:41.666615 15117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:41.668102 15091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:41.668265 15091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "40a4431d4f604537bcd77b969acef3da"
format_stamp: "Formatted at 2026-08-08 11:43:41 on dist-test-slave-39l8"
server_key: "0b2e2df6f3e9fbfb3ed61773e370559d"
server_key_iv: "6617f54eb48104af58ce0a0700c62b70"
server_key_version: "encryptionkey@0"
I20260808 11:43:41.668474 15091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:41.703395 15091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:41.707448 15091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:41.707724 15091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:41.708752 15091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:43:41.710039 15091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:41.710112 15091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:41.710201 15091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:41.710245 15091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:41.743302 15091 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:40071
I20260808 11:43:41.743403 15289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:40071 every 8 connection(s)
I20260808 11:43:41.744681 15091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:43:41.747438 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 15091
I20260808 11:43:41.747664 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:43:41.748152 13945 external_mini_cluster.cc:1428] Setting key 210407dcd9c3d1d114fc3d59c95a7fb7
Aug 08 11:43:41 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189421, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:41.765738 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:41.747411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:41355 (local address 127.13.158.126:46533)
0808 11:43:41.747649 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:41.747656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:41.749230 (+ 1574us) server_negotiation.cc:407] Connection header received
0808 11:43:41.751005 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:41.751010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:41.751103 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:41.751214 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:41.752692 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:41.753408 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:41.755058 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:41.755489 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:41.755637 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:41.759684 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:41.759689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:41.759692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:41.759724 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:41.762230 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:41.763240 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:41.763244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:41.763246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:41.763403 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:41.763963 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:41.763966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:41.763968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:41.764218 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:41.764384 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:41.765294 (+ 910us) server_negotiation.cc:299] Negotiation successful
0808 11:43:41.765471 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:43:41.767372 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:41.748007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46533 (local address 127.13.158.66:41355)
0808 11:43:41.748822 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:41.748866 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:41.750695 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:41.751449 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:41.751457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:41.751946 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:41.752518 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:41.752537 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:41.753577 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:41.753582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:41.754903 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:41.754916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:41.755734 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:41.755743 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:41.755967 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:41.756730 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:41.756756 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:41.759465 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:41.762373 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:41.762384 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:41.762402 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:41.763096 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:41.763526 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:41.763534 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:41.763541 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:41.763826 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:41.764354 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:41.764365 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:41.764883 (+ 518us) client_negotiation.cc:769] Sending connection context
0808 11:43:41.765199 (+ 316us) client_negotiation.cc:240] Negotiation successful
0808 11:43:41.765824 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":352,"threads_started":1}
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
I20260808 11:43:41.768859 15292 heartbeater.cc:347] Connected to a master server at 127.13.158.126:46533
I20260808 11:43:41.769129 15292 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:41.769924 15292 heartbeater.cc:511] Master 127.13.158.126:46533 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
I20260808 11:43:41.771490 14795 ts_manager.cc:194] Registered new tserver with Master: 40a4431d4f604537bcd77b969acef3da (127.13.158.66:40071)
I20260808 11:43:41.772321 14795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:41355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:43:41.793424 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46533
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:42.124893 15301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:42.125222 15301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:42.125393 15301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:42.132625 15301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:42.132805 15301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:42.132980 15301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:43:42.142351 15301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46533
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:42.143656 15301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:42.144937 15301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:42.157644 15311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:42.157745 15315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:42.157721 15312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:42.159198 15301 server_base.cc:1034] running on GCE node
I20260808 11:43:42.159722 15301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:42.160578 15301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:42.161809 15301 hybrid_clock.cc:648] HybridClock initialized: now 1786189422161772 us; error 58 us; skew 500 ppm
I20260808 11:43:42.162139 15301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:42 dist-test-slave-39l8 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189422, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:42.165516 15301 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:43:42.167218 15301 webserver.cc:460] Webserver started at http://127.13.158.67:45693/ using document root <none> and password file <none>
I20260808 11:43:42.167953 15301 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:42.168102 15301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:42.168431 15301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:42.171442 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "82521383d2894e6ca31f8c6211ce2406"
format_stamp: "Formatted at 2026-08-08 11:43:42 on dist-test-slave-39l8"
server_key: "08e5dda15244f43e1efbc29f05b0617c"
server_key_iv: "2c650ba81fbb3f7a79c435a2539de3a4"
server_key_version: "encryptionkey@0"
I20260808 11:43:42.172318 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "82521383d2894e6ca31f8c6211ce2406"
format_stamp: "Formatted at 2026-08-08 11:43:42 on dist-test-slave-39l8"
server_key: "08e5dda15244f43e1efbc29f05b0617c"
server_key_iv: "2c650ba81fbb3f7a79c435a2539de3a4"
server_key_version: "encryptionkey@0"
I20260808 11:43:42.177593 15301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:43:42.180886 15325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:42.182109 15301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:43:42.182296 15301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "82521383d2894e6ca31f8c6211ce2406"
format_stamp: "Formatted at 2026-08-08 11:43:42 on dist-test-slave-39l8"
server_key: "08e5dda15244f43e1efbc29f05b0617c"
server_key_iv: "2c650ba81fbb3f7a79c435a2539de3a4"
server_key_version: "encryptionkey@0"
I20260808 11:43:42.182503 15301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:42.205606 15301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:42.209446 15301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:42.209762 15301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:42.210672 15301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:43:42.211987 15301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:42.212131 15301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:42.212280 15301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:42.212354 15301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:42.246255 15301 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:37017
I20260808 11:43:42.246373 15500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:37017 every 8 connection(s)
I20260808 11:43:42.247599 15301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:43:42.251369 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 15301
I20260808 11:43:42.251587 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:43:42.252170 13945 external_mini_cluster.cc:1428] Setting key 22cff78b786ede1434d1e8b52f9a4b56
Aug 08 11:43:42 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189422, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:42.266872 15084 heartbeater.cc:503] Master 127.13.158.126:46533 was elected leader, sending a full tablet report...
I20260808 11:43:42.267318 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.250132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:40201 (local address 127.13.158.126:46533)
0808 11:43:42.250318 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.250323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.251503 (+ 1180us) server_negotiation.cc:407] Connection header received
0808 11:43:42.254077 (+ 2574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.254086 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.254175 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.254255 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:42.256005 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.256715 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.258137 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.258539 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.258628 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.261916 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:42.261920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:42.261923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:42.261952 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:42.264300 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.264935 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.264938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.264940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.265057 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.265455 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.265458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.265460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.265747 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:42.265926 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.266782 (+ 856us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.266992 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:43:42.268517 15505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.250488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46533 (local address 127.13.158.67:40201)
0808 11:43:42.251333 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:42.251385 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:42.253779 (+ 2394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:42.254499 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:42.254506 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:42.255204 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:42.255857 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.255874 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.256842 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.256846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.258015 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.258027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.258787 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.258798 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.259026 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.259715 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:42.259736 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:42.261735 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:42.264419 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:42.264426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:42.264436 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:42.264831 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:42.265147 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:42.265150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:42.265152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:42.265356 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:42.265912 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:42.265917 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:42.266304 (+ 387us) client_negotiation.cc:769] Sending connection context
0808 11:43:42.266627 (+ 323us) client_negotiation.cc:240] Negotiation successful
0808 11:43:42.267164 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":305,"threads_started":1}
I20260808 11:43:42.269877 15502 heartbeater.cc:347] Connected to a master server at 127.13.158.126:46533
I20260808 11:43:42.270193 15502 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:42.270949 15502 heartbeater.cc:511] Master 127.13.158.126:46533 requested a full tablet report, sending...
I20260808 11:43:42.272297 14795 ts_manager.cc:194] Registered new tserver with Master: 82521383d2894e6ca31f8c6211ce2406 (127.13.158.67:37017)
I20260808 11:43:42.273114 14795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:40201
I20260808 11:43:42.279923 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:43:42.299405 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.286414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.13.158.126:46533)
0808 11:43:42.286586 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.286590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.287192 (+ 602us) server_negotiation.cc:407] Connection header received
0808 11:43:42.287389 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.287396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.287480 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.287588 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:42.289374 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.290119 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.291518 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.291948 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.292097 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.294268 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:42.294274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:42.294284 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:42.294314 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:42.296737 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.297396 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.297402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.297405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.297537 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.297947 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.297952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.297955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.298260 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:42.298521 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.298884 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.299089 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:43:42.305699 14795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:43:42.312477 14795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:42.336301 15525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.325280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40428 (local address 127.13.158.66:40071)
0808 11:43:42.326312 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.326324 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.326360 (+ 36us) server_negotiation.cc:407] Connection header received
0808 11:43:42.326481 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.326492 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.326824 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.328048 (+ 1224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:42.328360 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.330497 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.333572 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.334481 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.334675 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.334925 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.335833 (+ 908us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.336013 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":772,"threads_started":1}
I20260808 11:43:42.336898 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.324665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:40071 (local address 127.0.0.1:40428)
0808 11:43:42.325418 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:42.325482 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:42.325677 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:42.327178 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:42.327189 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:42.327233 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:42.327722 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.327739 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.331376 (+ 3637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.331382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.333400 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.333413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.334780 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.334787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.334982 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.335045 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 11:43:42.335196 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 11:43:42.335381 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":6126,"threads_started":1}
I20260808 11:43:42.340758 15188 tablet_service.cc:1467] Processing CreateTablet for tablet 824943db8cb940e6b9208a77029dc459 (DEFAULT_TABLE table=test-table [id=5dbb16e9d6274eed94464273646a1cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:42.342393 15188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824943db8cb940e6b9208a77029dc459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:42.343262 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.325811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42981 (local address 127.0.0.1:43486)
0808 11:43:42.335877 (+ 10066us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:42.335895 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:42.336009 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:42.336888 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:42.336892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:42.336918 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:42.337354 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.337367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.339149 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.339154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.340596 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.340609 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.342377 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.342383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.342548 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.342593 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 11:43:42.342726 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 11:43:42.342856 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9882,"mutex_wait_us":1,"thread_start_us":9726,"threads_started":1}
I20260808 11:43:42.343403 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.326470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43486 (local address 127.13.158.65:42981)
0808 11:43:42.331470 (+ 5000us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.331480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.336129 (+ 4649us) server_negotiation.cc:407] Connection header received
0808 11:43:42.336239 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.336246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.336468 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.336629 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:42.337542 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.339009 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.340937 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.342139 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.342290 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.342549 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.342936 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.343099 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4824,"thread_start_us":4649,"threads_started":1}
I20260808 11:43:42.343820 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.324745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33570 (local address 127.13.158.67:37017)
0808 11:43:42.335416 (+ 10671us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.335423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.335453 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:43:42.335562 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.335570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.335813 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.336682 (+ 869us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:42.336978 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.338603 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.341089 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.342447 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.342568 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.342822 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.343360 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.343529 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10506,"thread_start_us":10378,"threads_started":1}
I20260808 11:43:42.344185 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.324702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37017 (local address 127.0.0.1:33570)
0808 11:43:42.331072 (+ 6370us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:42.331094 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:42.331217 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:42.336031 (+ 4814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:42.336036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:42.336063 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:42.336458 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.336470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.338794 (+ 2324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.338803 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.340901 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.340916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.342731 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.342738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.342894 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.342924 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:43:42.343017 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 11:43:42.343143 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6228,"thread_start_us":331,"threads_started":1}
I20260808 11:43:42.347021 14987 tablet_service.cc:1467] Processing CreateTablet for tablet 824943db8cb940e6b9208a77029dc459 (DEFAULT_TABLE table=test-table [id=5dbb16e9d6274eed94464273646a1cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:42.347106 15404 tablet_service.cc:1467] Processing CreateTablet for tablet 824943db8cb940e6b9208a77029dc459 (DEFAULT_TABLE table=test-table [id=5dbb16e9d6274eed94464273646a1cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:42.348438 15404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824943db8cb940e6b9208a77029dc459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:42.348469 14987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824943db8cb940e6b9208a77029dc459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:42.359191 15529 tablet_bootstrap.cc:492] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406: Bootstrap starting.
I20260808 11:43:42.360134 15528 tablet_bootstrap.cc:492] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da: Bootstrap starting.
I20260808 11:43:42.362564 15532 tablet_bootstrap.cc:492] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1: Bootstrap starting.
I20260808 11:43:42.363955 15528 tablet_bootstrap.cc:654] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:42.364912 15529 tablet_bootstrap.cc:654] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:42.365319 15528 log.cc:824] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:42.365504 15532 tablet_bootstrap.cc:654] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:42.366191 15529 log.cc:824] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:42.366794 15532 log.cc:824] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:42.368654 15529 tablet_bootstrap.cc:492] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406: No bootstrap required, opened a new log
I20260808 11:43:42.368881 15528 tablet_bootstrap.cc:492] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da: No bootstrap required, opened a new log
I20260808 11:43:42.368959 15529 ts_tablet_manager.cc:1397] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 11:43:42.369170 15528 ts_tablet_manager.cc:1397] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 11:43:42.371061 15532 tablet_bootstrap.cc:492] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1: No bootstrap required, opened a new log
I20260808 11:43:42.371325 15532 ts_tablet_manager.cc:1397] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260808 11:43:42.376420 15529 raft_consensus.cc:348] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.376773 15529 raft_consensus.cc:374] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:42.376883 15529 raft_consensus.cc:729] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82521383d2894e6ca31f8c6211ce2406, State: Initialized, Role: FOLLOWER
I20260808 11:43:42.377465 15529 consensus_queue.cc:260] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.378579 15502 heartbeater.cc:503] Master 127.13.158.126:46533 was elected leader, sending a full tablet report...
I20260808 11:43:42.379154 15529 ts_tablet_manager.cc:1428] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 11:43:42.380357 15528 raft_consensus.cc:348] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.380784 15528 raft_consensus.cc:374] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:42.380885 15528 raft_consensus.cc:729] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a4431d4f604537bcd77b969acef3da, State: Initialized, Role: FOLLOWER
I20260808 11:43:42.381426 15532 raft_consensus.cc:348] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.381752 15532 raft_consensus.cc:374] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:42.381562 15528 consensus_queue.cc:260] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.381850 15532 raft_consensus.cc:729] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c0a02f591e046268d7ea31b867d23d1, State: Initialized, Role: FOLLOWER
I20260808 11:43:42.382808 15292 heartbeater.cc:503] Master 127.13.158.126:46533 was elected leader, sending a full tablet report...
I20260808 11:43:42.383350 15528 ts_tablet_manager.cc:1428] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 11:43:42.383586 15532 consensus_queue.cc:260] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.385381 15532 ts_tablet_manager.cc:1428] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260808 11:43:42.486521 15085 tablet.cc:2369] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:43:42.496722 15293 tablet.cc:2369] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:43:42.499147 15503 tablet.cc:2369] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:43:42.819802 15541 raft_consensus.cc:482] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:42.820155 15541 raft_consensus.cc:504] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.821939 15541 leader_election.cc:288] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82521383d2894e6ca31f8c6211ce2406 (127.13.158.67:37017), 8c0a02f591e046268d7ea31b867d23d1 (127.13.158.65:42981)
I20260808 11:43:42.823551 15542 raft_consensus.cc:482] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:42.823971 15542 raft_consensus.cc:504] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.826531 15542 leader_election.cc:288] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82521383d2894e6ca31f8c6211ce2406 (127.13.158.67:37017), 40a4431d4f604537bcd77b969acef3da (127.13.158.66:40071)
I20260808 11:43:42.831272 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.822373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:47595 (local address 127.13.158.67:37017)
0808 11:43:42.822550 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.822555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.823200 (+ 645us) server_negotiation.cc:407] Connection header received
0808 11:43:42.823729 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.823735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.823818 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.823889 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:42.824711 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.825687 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.828769 (+ 3082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.830035 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.830164 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.830249 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.830857 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.830990 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:43:42.831439 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.823456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:57713 (local address 127.13.158.65:42981)
0808 11:43:42.823670 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.823675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.824088 (+ 413us) server_negotiation.cc:407] Connection header received
0808 11:43:42.824287 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.824291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.824363 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.824460 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:42.825124 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.826189 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.829106 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.830345 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.830467 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.830537 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.831094 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.831209 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 11:43:42.832154 15543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.822341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37017 (local address 127.13.158.66:47595)
0808 11:43:42.823083 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:42.823104 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:42.823518 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:42.824095 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:42.824098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:42.824129 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:42.824510 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.824523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.825848 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.825853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.828596 (+ 2743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.828611 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.830437 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.830444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.830635 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.830676 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 11:43:42.831543 (+ 867us) client_negotiation.cc:240] Negotiation successful
0808 11:43:42.831677 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":97,"thread_start_us":403,"threads_started":1}
I20260808 11:43:42.832698 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.823251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42981 (local address 127.13.158.66:57713)
0808 11:43:42.823953 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:42.823974 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:42.824126 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:42.824494 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:42.824499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:42.824524 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:42.824943 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.824957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.826358 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.826363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.828941 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.828955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.830751 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.830760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.830927 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.830957 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:43:42.831567 (+ 610us) client_negotiation.cc:240] Negotiation successful
0808 11:43:42.831677 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":361,"thread_start_us":420,"threads_started":1}
I20260808 11:43:42.833361 15434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824943db8cb940e6b9208a77029dc459" candidate_uuid: "40a4431d4f604537bcd77b969acef3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82521383d2894e6ca31f8c6211ce2406" is_pre_election: true
I20260808 11:43:42.833941 15434 raft_consensus.cc:2454] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a4431d4f604537bcd77b969acef3da in term 0.
I20260808 11:43:42.833992 15016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824943db8cb940e6b9208a77029dc459" candidate_uuid: "40a4431d4f604537bcd77b969acef3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0a02f591e046268d7ea31b867d23d1" is_pre_election: true
I20260808 11:43:42.834429 15016 raft_consensus.cc:2454] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a4431d4f604537bcd77b969acef3da in term 0.
I20260808 11:43:42.834607 15122 leader_election.cc:302] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a4431d4f604537bcd77b969acef3da, 82521383d2894e6ca31f8c6211ce2406; no voters:
I20260808 11:43:42.835307 15541 raft_consensus.cc:2781] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:42.835554 15541 raft_consensus.cc:482] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:42.835693 15541 raft_consensus.cc:3037] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:42.839084 15541 raft_consensus.cc:504] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.839831 15541 leader_election.cc:288] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [CANDIDATE]: Term 1 election: Requested vote from peers 82521383d2894e6ca31f8c6211ce2406 (127.13.158.67:37017), 8c0a02f591e046268d7ea31b867d23d1 (127.13.158.65:42981)
I20260808 11:43:42.839938 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.826407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:47699 (local address 127.13.158.67:37017)
0808 11:43:42.831499 (+ 5092us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.831505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.832348 (+ 843us) server_negotiation.cc:407] Connection header received
0808 11:43:42.833145 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.833151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.833263 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.833366 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:42.834228 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.835281 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.837850 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.838801 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.838971 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.839041 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.839481 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.839620 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4989,"thread_start_us":6819,"threads_started":1}
I20260808 11:43:42.840333 15434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824943db8cb940e6b9208a77029dc459" candidate_uuid: "40a4431d4f604537bcd77b969acef3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82521383d2894e6ca31f8c6211ce2406"
I20260808 11:43:42.840547 15434 raft_consensus.cc:3037] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:42.840515 15016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824943db8cb940e6b9208a77029dc459" candidate_uuid: "40a4431d4f604537bcd77b969acef3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0a02f591e046268d7ea31b867d23d1"
I20260808 11:43:42.840739 15525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.826928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:37747 (local address 127.13.158.66:40071)
0808 11:43:42.827124 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.827132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.833933 (+ 6801us) server_negotiation.cc:407] Connection header received
0808 11:43:42.834208 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.834212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.834290 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.834352 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:42.835335 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.836206 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.838082 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.839405 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.839541 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.839610 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.840379 (+ 769us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.840505 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 11:43:42.840749 15016 raft_consensus.cc:3037] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:42.841034 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.826211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:40071 (local address 127.13.158.65:37747)
0808 11:43:42.833824 (+ 7613us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:42.833848 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:42.834041 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:42.834550 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:42.834555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:42.834581 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:42.835130 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.835144 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.836356 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.836360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.837918 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.837933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.839890 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.839897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.840074 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.840115 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 11:43:42.840215 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 11:43:42.840348 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7471,"thread_start_us":8584,"threads_started":1}
I20260808 11:43:42.840751 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.826020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37017 (local address 127.13.158.65:47699)
0808 11:43:42.832246 (+ 6226us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:42.832268 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:42.832947 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:42.833548 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:42.833557 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:42.833582 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:42.834030 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.834043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.835444 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.835449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.837604 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:42.837622 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.839168 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.839172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:42.839280 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.839338 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 11:43:42.839440 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 11:43:42.839563 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6071,"mutex_wait_us":1,"thread_start_us":5863,"threads_started":1}
I20260808 11:43:42.842077 15222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824943db8cb940e6b9208a77029dc459" candidate_uuid: "8c0a02f591e046268d7ea31b867d23d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a4431d4f604537bcd77b969acef3da" is_pre_election: true
I20260808 11:43:42.842276 15433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824943db8cb940e6b9208a77029dc459" candidate_uuid: "8c0a02f591e046268d7ea31b867d23d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82521383d2894e6ca31f8c6211ce2406" is_pre_election: true
I20260808 11:43:42.842558 15222 raft_consensus.cc:2379] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c0a02f591e046268d7ea31b867d23d1 in current term 1: Already voted for candidate 40a4431d4f604537bcd77b969acef3da in this term.
I20260808 11:43:42.843494 14910 leader_election.cc:302] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c0a02f591e046268d7ea31b867d23d1; no voters: 40a4431d4f604537bcd77b969acef3da, 82521383d2894e6ca31f8c6211ce2406
I20260808 11:43:42.844048 15434 raft_consensus.cc:2454] T 824943db8cb940e6b9208a77029dc459 P 82521383d2894e6ca31f8c6211ce2406 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a4431d4f604537bcd77b969acef3da in term 1.
I20260808 11:43:42.844156 15016 raft_consensus.cc:2454] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a4431d4f604537bcd77b969acef3da in term 1.
I20260808 11:43:42.844295 15542 raft_consensus.cc:2726] T 824943db8cb940e6b9208a77029dc459 P 8c0a02f591e046268d7ea31b867d23d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 11:43:42.844457 15122 leader_election.cc:302] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a4431d4f604537bcd77b969acef3da, 82521383d2894e6ca31f8c6211ce2406; no voters:
I20260808 11:43:42.844741 15541 raft_consensus.cc:2781] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:42.845145 15541 raft_consensus.cc:686] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [term 1 LEADER]: Becoming Leader. State: Replica: 40a4431d4f604537bcd77b969acef3da, State: Running, Role: LEADER
I20260808 11:43:42.845600 15541 consensus_queue.cc:237] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } }
I20260808 11:43:42.850243 14794 catalog_manager.cc:5495] T 824943db8cb940e6b9208a77029dc459 P 40a4431d4f604537bcd77b969acef3da reported cstate change: term changed from 0 to 1, leader changed from <none> to 40a4431d4f604537bcd77b969acef3da (127.13.158.66). New cstate: current_term: 1 leader_uuid: "40a4431d4f604537bcd77b969acef3da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82521383d2894e6ca31f8c6211ce2406" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a4431d4f604537bcd77b969acef3da" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 40071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c0a02f591e046268d7ea31b867d23d1" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42981 } health_report { overall_health: UNKNOWN } } }
Aug 08 11:43:42 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:43:42.902024 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.887724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.13.158.65:42981)
0808 11:43:42.887951 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.887960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.888318 (+ 358us) server_negotiation.cc:407] Connection header received
0808 11:43:42.888585 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.888592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.888688 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.888773 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:42.890077 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.890869 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.892141 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.892531 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.892621 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.896586 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:42.896600 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:42.896617 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:42.896650 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:42.899419 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.900098 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.900105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.900107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.900231 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.900671 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.900674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.900676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.901017 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:42.901233 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.901450 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.901689 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 11:43:42 dist-test-slave-39l8 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:43:42 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:43:42.950775 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.936560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.13.158.65:42981)
0808 11:43:42.936839 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.936849 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.937186 (+ 337us) server_negotiation.cc:407] Connection header received
0808 11:43:42.937408 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.937412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.937487 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.937611 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:42.939031 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.939746 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.940973 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.941435 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.941588 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.945515 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:42.945519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:42.945523 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:42.945557 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:42.948180 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.948806 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.948813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.948816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.948951 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.949350 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.949355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.949358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.949612 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:42.949873 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.950140 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.950362 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 11:43:42 dist-test-slave-39l8 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:43:42 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:43:42.997227 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:42.983740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.13.158.65:42981)
0808 11:43:42.983940 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:42.983948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:42.984269 (+ 321us) server_negotiation.cc:407] Connection header received
0808 11:43:42.984493 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:42.984497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:42.984562 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:42.984665 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:42.986062 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.986913 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.988175 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:42.988597 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:42.988770 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:42.992455 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:42.992460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:42.992463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:42.992494 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:42.994968 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.995611 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.995615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.995617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.995732 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:42.996026 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:42.996029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:42.996031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:42.996297 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:42.996548 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:42.996822 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 11:43:42.996985 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 11:43:42.999136 14950 tablet_service.cc:2908] Rejecting scan request for tablet 824943db8cb940e6b9208a77029dc459: Uninitialized: clean time has not yet been initialized
I20260808 11:43:43.002764 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14885
I20260808 11:43:43.015563 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:43.003705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.13.158.65:42981)
0808 11:43:43.003860 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:43.003865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:43.004287 (+ 422us) server_negotiation.cc:407] Connection header received
0808 11:43:43.004402 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:43.004406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:43.004470 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:43.004549 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:43.005709 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:43.006339 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:43.007504 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:43.007907 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:43.007999 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:43.010131 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:43.010135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:43.010138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:43.010167 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:43.012972 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:43.013617 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:43.013634 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:43.013636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:43.013758 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:43.014097 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:43.014100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:43.014102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:43.014386 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:43.014594 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:43.014977 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 11:43:43.015272 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:43:43.016290 15078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:43.106833 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14885
I20260808 11:43:43.131528 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 15091
Aug 08 11:43:43 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:43:43.145531 15525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:43.132300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.13.158.66:40071)
0808 11:43:43.132473 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:43.132477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:43.132540 (+ 63us) server_negotiation.cc:407] Connection header received
0808 11:43:43.132714 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:43.132718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:43.132783 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:43.132867 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:43.134029 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:43.134855 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:43.136060 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:43.136423 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:43.136510 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:43.140390 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:43.140402 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:43.140417 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:43.140448 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:43.143246 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:43.143827 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:43.143834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:43.143836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:43.143958 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:43.144315 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:43.144319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:43.144321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:43.144650 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:43.144879 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:43.145117 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 11:43:43.145271 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:43:43.146224 15285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:43.235095 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 15091
W20260808 11:43:43.260222 15333 connection.cc:535] server connection from 127.13.158.66:47595 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 11:43:43.260627 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 15301
Aug 08 11:43:43 dist-test-slave-39l8 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:43:43.274398 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:43.261417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33580 (local address 127.13.158.67:37017)
0808 11:43:43.261572 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:43.261576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:43.261593 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:43:43.261669 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:43.261673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:43.261738 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:43.261816 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:43.263033 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:43.263755 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:43.264923 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:43.265320 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:43.265405 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:43.269468 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:43.269479 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:43.269493 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:43.269522 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:43.272197 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:43.272824 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:43.272830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:43.272832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:43.272946 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:43.273275 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:43.273277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:43.273279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:43.273592 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:43.273817 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:43.274011 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 11:43:43.274169 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:43:43.275090 15495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:43.361258 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 15301
I20260808 11:43:43.388751 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14746
I20260808 11:43:43.390100 14845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:43.496591 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14746
2026-08-08T11:43:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4718 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:43 dist-test-slave-39l8 krb5kdc[15583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:43 dist-test-slave-39l8 krb5kdc[15583](info): set up 2 sockets
Aug 08 11:43:43 dist-test-slave-39l8 krb5kdc[15583](info): commencing operation
krb5kdc: starting...
W20260808 11:43:44.777463 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.003s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:43:44 dist-test-slave-39l8 krb5kdc[15583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:43:44.971527 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:39087
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:39087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:45.302440 15603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:45.302806 15603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:45.302965 15603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:45.310321 15603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:45.310428 15603 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 11:43:45.310469 15603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:45.310523 15603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:45.310599 15603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:45.310725 15603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:45.320758 15603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:39087
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:39087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:45.322093 15603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:45.323473 15603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:45.335224 15611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:45.335235 15609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:45.335469 15608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:45.336216 15603 server_base.cc:1034] running on GCE node
I20260808 11:43:45.336975 15603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:45.338316 15603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:45.339565 15603 hybrid_clock.cc:648] HybridClock initialized: now 1786189425339512 us; error 48 us; skew 500 ppm
I20260808 11:43:45.339921 15603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:45 dist-test-slave-39l8 krb5kdc[15583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189425, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:45.343003 15603 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:43:45.344408 15603 webserver.cc:460] Webserver started at http://127.13.158.126:38119/ using document root <none> and password file <none>
I20260808 11:43:45.345094 15603 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:45.345237 15603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:45.345567 15603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:45.348582 15603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "ebafcb85a60344918fae55632a84dbd4"
format_stamp: "Formatted at 2026-08-08 11:43:45 on dist-test-slave-39l8"
server_key: "fcc0ffc6d8c075cf29e8294db3bd8faa"
server_key_iv: "2fbe6eef37b1eab224a619bd0db3edc4"
server_key_version: "encryptionkey@0"
I20260808 11:43:45.349357 15603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "ebafcb85a60344918fae55632a84dbd4"
format_stamp: "Formatted at 2026-08-08 11:43:45 on dist-test-slave-39l8"
server_key: "fcc0ffc6d8c075cf29e8294db3bd8faa"
server_key_iv: "2fbe6eef37b1eab224a619bd0db3edc4"
server_key_version: "encryptionkey@0"
I20260808 11:43:45.354276 15603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 11:43:45.357529 15627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:45.358815 15603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:45.359000 15603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "ebafcb85a60344918fae55632a84dbd4"
format_stamp: "Formatted at 2026-08-08 11:43:45 on dist-test-slave-39l8"
server_key: "fcc0ffc6d8c075cf29e8294db3bd8faa"
server_key_iv: "2fbe6eef37b1eab224a619bd0db3edc4"
server_key_version: "encryptionkey@0"
I20260808 11:43:45.359216 15603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:45.388195 15603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:45.392151 15603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:45.392436 15603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:45.414318 15603 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:39087
I20260808 11:43:45.414326 15709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:39087 every 8 connection(s)
I20260808 11:43:45.415664 15603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:45.419342 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:45.422086 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 15603
I20260808 11:43:45.422330 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:43:45.423004 13945 external_mini_cluster.cc:1428] Setting key d6ead5ecf2ea5fe503c203679997a580
I20260808 11:43:45.432691 15712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4: Bootstrap starting.
I20260808 11:43:45.436476 15712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:45.437493 15712 log.cc:824] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:43:45 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:45.440608 15712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4: No bootstrap required, opened a new log
I20260808 11:43:45.447058 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:45.425143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58744 (local address 127.13.158.126:39087)
0808 11:43:45.425973 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:45.425989 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:45.426040 (+ 51us) server_negotiation.cc:407] Connection header received
0808 11:43:45.428357 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:45.428395 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:45.428908 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:45.429374 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:45.430680 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:45.432599 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:45.433899 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:45.435053 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:45.435251 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:45.439641 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:45.439648 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:45.439663 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:45.439706 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:45.442999 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:45.443811 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:45.443817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:45.443823 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:45.443977 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:45.444371 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:45.444374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:45.444376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:45.444881 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:45.445115 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:45.445563 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 11:43:45.445963 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":333,"threads_started":1}
I20260808 11:43:45.449287 15712 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebafcb85a60344918fae55632a84dbd4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39087 } }
I20260808 11:43:45.449657 15712 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:45.449787 15712 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebafcb85a60344918fae55632a84dbd4, State: Initialized, Role: FOLLOWER
I20260808 11:43:45.450387 15712 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebafcb85a60344918fae55632a84dbd4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39087 } }
I20260808 11:43:45.450620 15712 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:45.450774 15712 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:45.450943 15712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:45.453184 15712 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebafcb85a60344918fae55632a84dbd4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39087 } }
I20260808 11:43:45.453670 15712 leader_election.cc:302] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebafcb85a60344918fae55632a84dbd4; no voters:
I20260808 11:43:45.454212 15712 leader_election.cc:288] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:45.454383 15719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:45.455260 15719 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [term 1 LEADER]: Becoming Leader. State: Replica: ebafcb85a60344918fae55632a84dbd4, State: Running, Role: LEADER
I20260808 11:43:45.456103 15719 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebafcb85a60344918fae55632a84dbd4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39087 } }
I20260808 11:43:45.456581 15712 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:45.458971 15720 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebafcb85a60344918fae55632a84dbd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebafcb85a60344918fae55632a84dbd4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39087 } } }
I20260808 11:43:45.459471 15720 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:45.458942 15721 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebafcb85a60344918fae55632a84dbd4. Latest consensus state: current_term: 1 leader_uuid: "ebafcb85a60344918fae55632a84dbd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebafcb85a60344918fae55632a84dbd4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39087 } } }
I20260808 11:43:45.459748 15721 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:45.461390 15729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:45.466910 15729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:45.475466 15729 catalog_manager.cc:1347] Generated new cluster ID: d30577e5a4a74f748c51ebf85a824bc4
I20260808 11:43:45.475615 15729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:45.498364 15729 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:45.500306 15729 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:45.514591 15729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4: Generated new TSK 0
I20260808 11:43:45.515625 15729 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:43:45.564541 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39087
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:45.898142 15750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:45.898422 15750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:45.898584 15750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:45.905763 15750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:45.905864 15750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:45.905939 15750 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:43:45.906068 15750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:43:45.915867 15750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:45.917173 15750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:45.918527 15750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:45.932220 15760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:45.932168 15766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:45.932256 15759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:45.932919 15750 server_base.cc:1034] running on GCE node
I20260808 11:43:45.933545 15750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:45.934543 15750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:45.935811 15750 hybrid_clock.cc:648] HybridClock initialized: now 1786189425935766 us; error 68 us; skew 500 ppm
I20260808 11:43:45.936123 15750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:45 dist-test-slave-39l8 krb5kdc[15583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189425, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:45.943208 15750 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:43:45.944736 15750 webserver.cc:460] Webserver started at http://127.13.158.65:43561/ using document root <none> and password file <none>
I20260808 11:43:45.945473 15750 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:45.945613 15750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:45.946002 15750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:45.949148 15750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "b1d3947f42fd4982b8794c66cb154cf8"
format_stamp: "Formatted at 2026-08-08 11:43:45 on dist-test-slave-39l8"
server_key: "b47753202293b8d6952da9a2b66dbe7a"
server_key_iv: "1c37bd486f3087831cffeeb883f0e49a"
server_key_version: "encryptionkey@0"
I20260808 11:43:45.949878 15750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "b1d3947f42fd4982b8794c66cb154cf8"
format_stamp: "Formatted at 2026-08-08 11:43:45 on dist-test-slave-39l8"
server_key: "b47753202293b8d6952da9a2b66dbe7a"
server_key_iv: "1c37bd486f3087831cffeeb883f0e49a"
server_key_version: "encryptionkey@0"
I20260808 11:43:45.954962 15750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260808 11:43:45.958639 15774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:45.960047 15750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:43:45.960230 15750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "b1d3947f42fd4982b8794c66cb154cf8"
format_stamp: "Formatted at 2026-08-08 11:43:45 on dist-test-slave-39l8"
server_key: "b47753202293b8d6952da9a2b66dbe7a"
server_key_iv: "1c37bd486f3087831cffeeb883f0e49a"
server_key_version: "encryptionkey@0"
I20260808 11:43:45.960464 15750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:45.978785 15750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:45.982865 15750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:45.983156 15750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:45.985954 15750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:45.986101 15750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:45.986282 15750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:45.986373 15750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:43:46 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189425, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:46.012806 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:45.986353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:36921 (local address 127.13.158.126:39087)
0808 11:43:45.986928 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:45.986935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:45.988232 (+ 1297us) server_negotiation.cc:407] Connection header received
0808 11:43:45.991337 (+ 3105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:45.991358 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:45.991448 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:45.991575 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:45.994397 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:45.995490 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:45.998894 (+ 3404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:45.999392 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:45.999541 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:46.005090 (+ 5549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:46.005098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:46.005106 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:46.005155 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:46.009191 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:46.010237 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:46.010242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:46.010246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:46.010375 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:46.010925 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:46.010932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:46.010938 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:46.011281 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:46.011433 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:46.012218 (+ 785us) server_negotiation.cc:299] Negotiation successful
0808 11:43:46.012435 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":248,"threads_started":1}
I20260808 11:43:46.013980 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:45.987059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39087 (local address 127.13.158.65:36921)
0808 11:43:45.988025 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:45.988099 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:45.990954 (+ 2855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:45.991906 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:45.991922 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:45.992956 (+ 1034us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:45.994054 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:45.994114 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:45.995724 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:45.995733 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:45.998623 (+ 2890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:45.998645 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:45.999878 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:45.999882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:46.000185 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:46.001072 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:46.001115 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:46.004819 (+ 3704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:46.009359 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:46.009372 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:46.009394 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:46.010092 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:46.010491 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:46.010496 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:46.010500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:46.010754 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:46.011413 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:46.011420 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:46.011836 (+ 416us) client_negotiation.cc:769] Sending connection context
0808 11:43:46.012125 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 11:43:46.012656 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":345,"threads_started":1}
I20260808 11:43:46.033170 15750 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:39157
I20260808 11:43:46.033187 15944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:39157 every 8 connection(s)
I20260808 11:43:46.034901 15750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:43:46.037016 15945 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39087
I20260808 11:43:46.037247 15945 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:46.038003 15945 heartbeater.cc:511] Master 127.13.158.126:39087 requested a full tablet report, sending...
I20260808 11:43:46.039384 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:46.022886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:35375 (local address 127.13.158.126:39087)
0808 11:43:46.023079 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:46.023086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:46.023956 (+ 870us) server_negotiation.cc:407] Connection header received
0808 11:43:46.024222 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:46.024233 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:46.024294 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:46.024350 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:46.025396 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.026630 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.029281 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.030001 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.030163 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:46.032667 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:46.032674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:46.032678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:46.032714 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:46.036488 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:46.037243 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:46.037250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:46.037253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:46.037422 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:46.037812 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:46.037818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:46.037821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:46.038105 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:46.038296 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:46.038739 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 11:43:46.038967 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:43:46.039402 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:46.023025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39087 (local address 127.13.158.65:35375)
0808 11:43:46.023801 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:46.023825 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:46.024010 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:46.024540 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:46.024543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:46.024955 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:46.025249 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:46.025258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.026819 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.026828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:46.029057 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:46.029076 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.030231 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.030242 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:46.030466 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:46.031291 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:46.031308 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:46.032432 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:46.036605 (+ 4173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:46.036612 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:46.036618 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:46.037135 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:46.037548 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:46.037555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:46.037576 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:46.037701 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:46.038253 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:46.038261 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:46.038454 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 11:43:46.038627 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 11:43:46.038972 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":444,"threads_started":1}
I20260808 11:43:46.040282 15650 ts_manager.cc:194] Registered new tserver with Master: b1d3947f42fd4982b8794c66cb154cf8 (127.13.158.65:39157)
I20260808 11:43:46.041683 15650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:36921
I20260808 11:43:46.044783 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 15750
I20260808 11:43:46.045012 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:43:46.045783 13945 external_mini_cluster.cc:1428] Setting key 9e5d790a08b992fcbf0783889c479450
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:43:46.088241 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39087
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:46.413498 15950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:46.413820 15950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:46.413995 15950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:46.421607 15950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:46.421768 15950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:46.421916 15950 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:43:46.422055 15950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:43:46.431566 15950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:46.432825 15950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:46.434166 15950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:46.447106 15963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:46.447196 15957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:46.447165 15958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:46.448022 15950 server_base.cc:1034] running on GCE node
I20260808 11:43:46.448596 15950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:46.449525 15950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:46.450832 15950 hybrid_clock.cc:648] HybridClock initialized: now 1786189426450763 us; error 45 us; skew 500 ppm
I20260808 11:43:46.451130 15950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:46 dist-test-slave-39l8 krb5kdc[15583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189426, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:46.454108 15950 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:43:46.455591 15950 webserver.cc:460] Webserver started at http://127.13.158.66:42801/ using document root <none> and password file <none>
I20260808 11:43:46.456281 15950 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:46.456446 15950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:46.456764 15950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:46.459797 15950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "634b716382f444fcad41dfc61960d950"
format_stamp: "Formatted at 2026-08-08 11:43:46 on dist-test-slave-39l8"
server_key: "08829cad25f6557549a020968a35c868"
server_key_iv: "ff4206efab90cf09df86eb15e5e7e64e"
server_key_version: "encryptionkey@0"
I20260808 11:43:46.460608 15950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "634b716382f444fcad41dfc61960d950"
format_stamp: "Formatted at 2026-08-08 11:43:46 on dist-test-slave-39l8"
server_key: "08829cad25f6557549a020968a35c868"
server_key_iv: "ff4206efab90cf09df86eb15e5e7e64e"
server_key_version: "encryptionkey@0"
I20260808 11:43:46.466099 15950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:43:46.469501 15977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:46.470826 15950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:46.470988 15950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "634b716382f444fcad41dfc61960d950"
format_stamp: "Formatted at 2026-08-08 11:43:46 on dist-test-slave-39l8"
server_key: "08829cad25f6557549a020968a35c868"
server_key_iv: "ff4206efab90cf09df86eb15e5e7e64e"
server_key_version: "encryptionkey@0"
I20260808 11:43:46.471217 15950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:46.487146 15950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:46.491109 15950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:46.491402 15950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:46.494128 15950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:46.494237 15950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:46.494416 15950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:46.494498 15950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:43:46 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189426, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:46.517623 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:46.494565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:38931 (local address 127.13.158.126:39087)
0808 11:43:46.494833 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:46.494844 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:46.496307 (+ 1463us) server_negotiation.cc:407] Connection header received
0808 11:43:46.498416 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:46.498424 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:46.498578 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:46.498741 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:46.501055 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.502149 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.504828 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.505254 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.505391 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:46.510628 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:46.510633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:46.510636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:46.510668 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:46.514086 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:46.515117 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:46.515124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:46.515132 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:46.515285 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:46.515798 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:46.515802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:46.515804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:46.516081 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:46.516241 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:46.517006 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 11:43:46.517258 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:43:46.518798 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:46.495226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39087 (local address 127.13.158.66:38931)
0808 11:43:46.496118 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:46.496167 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:46.498122 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:46.499015 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:46.499026 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:46.499849 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:46.500793 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:46.500826 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.502310 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.502318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:46.504659 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:46.504672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.505493 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.505503 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:46.505721 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:46.506499 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:46.506544 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:46.510361 (+ 3817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:46.514235 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:46.514248 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:46.514267 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:46.514985 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:46.515406 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:46.515411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:46.515415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:46.515617 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:46.516224 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:46.516232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:46.516604 (+ 372us) client_negotiation.cc:769] Sending connection context
0808 11:43:46.516873 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 11:43:46.517462 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":398,"threads_started":1}
I20260808 11:43:46.541215 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:46.524716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:34951 (local address 127.13.158.126:39087)
0808 11:43:46.524890 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:46.524898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:46.525648 (+ 750us) server_negotiation.cc:407] Connection header received
0808 11:43:46.525840 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:46.525843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:46.525903 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:46.525983 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:46.527647 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.528656 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.530630 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.531242 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.531373 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:46.534271 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:46.534275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:46.534279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:46.534310 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:46.537413 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:46.538617 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:46.538622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:46.538626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:46.538833 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:46.539355 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:46.539363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:46.539369 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:46.539894 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:46.540051 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:46.540824 (+ 773us) server_negotiation.cc:299] Negotiation successful
0808 11:43:46.540990 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:43:46.541788 16076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:46.524748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39087 (local address 127.13.158.66:34951)
0808 11:43:46.525504 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:46.525530 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:46.525716 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:46.526046 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:46.526054 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:46.526837 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:46.527437 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:46.527453 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.528799 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.528805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:46.530392 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:46.530404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:46.531640 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:46.531647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:46.531831 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:46.532771 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:46.532792 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:46.534037 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:46.537648 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:46.537653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:46.537658 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:46.538411 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:46.538993 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:46.538999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:46.539004 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:46.539186 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:46.540132 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:46.540144 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:46.540449 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 11:43:46.540700 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 11:43:46.541130 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":380,"threads_started":1}
I20260808 11:43:46.547523 15950 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:36753
I20260808 11:43:46.547566 16150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:36753 every 8 connection(s)
I20260808 11:43:46.548679 15950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:43:46.550618 16152 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39087
I20260808 11:43:46.550876 16152 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:46.551505 16152 heartbeater.cc:511] Master 127.13.158.126:39087 requested a full tablet report, sending...
I20260808 11:43:46.552876 15650 ts_manager.cc:194] Registered new tserver with Master: 634b716382f444fcad41dfc61960d950 (127.13.158.66:36753)
I20260808 11:43:46.553678 15650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:38931
I20260808 11:43:46.557483 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 15950
I20260808 11:43:46.557694 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:43:46.558127 13945 external_mini_cluster.cc:1428] Setting key 22a8b6870fdc7f5f638a0abca01fe242
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:43:46.601814 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39087
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:46.926533 16157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:46.926836 16157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:46.926995 16157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:46.934163 16157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:46.934283 16157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:46.934365 16157 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:43:46.934469 16157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:43:46.944070 16157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:46.945326 16157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:46.946645 16157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:46.959820 16172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:46.959899 16169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:46.960126 16168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:46.961288 16157 server_base.cc:1034] running on GCE node
I20260808 11:43:46.961948 16157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:46.963011 16157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:46.964270 16157 hybrid_clock.cc:648] HybridClock initialized: now 1786189426964203 us; error 44 us; skew 500 ppm
I20260808 11:43:46.964592 16157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:46 dist-test-slave-39l8 krb5kdc[15583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189426, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:46.967958 16157 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:43:46.969359 16157 webserver.cc:460] Webserver started at http://127.13.158.67:41097/ using document root <none> and password file <none>
I20260808 11:43:46.970018 16157 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:46.970180 16157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:46.970511 16157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:46.973416 16157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "6fcdba6532c44372a7614af195bbc6cd"
format_stamp: "Formatted at 2026-08-08 11:43:46 on dist-test-slave-39l8"
server_key: "2cb4644f64c0c99b2553c690b81444f5"
server_key_iv: "fa95b74d2df6a8d92d6e4daba8648b74"
server_key_version: "encryptionkey@0"
I20260808 11:43:46.974229 16157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "6fcdba6532c44372a7614af195bbc6cd"
format_stamp: "Formatted at 2026-08-08 11:43:46 on dist-test-slave-39l8"
server_key: "2cb4644f64c0c99b2553c690b81444f5"
server_key_iv: "fa95b74d2df6a8d92d6e4daba8648b74"
server_key_version: "encryptionkey@0"
I20260808 11:43:46.979408 16157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:43:46.982654 16184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:46.984021 16157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:43:46.984215 16157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "6fcdba6532c44372a7614af195bbc6cd"
format_stamp: "Formatted at 2026-08-08 11:43:46 on dist-test-slave-39l8"
server_key: "2cb4644f64c0c99b2553c690b81444f5"
server_key_iv: "fa95b74d2df6a8d92d6e4daba8648b74"
server_key_version: "encryptionkey@0"
I20260808 11:43:46.984452 16157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:47.002724 16157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:47.007160 16157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:47.007421 16157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:47.010776 16157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:47.010910 16157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 11:43:47.011086 16157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:47.011159 16157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:43:47 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189426, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:47.032450 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.010989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:32933 (local address 127.13.158.126:39087)
0808 11:43:47.011199 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.011206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.012608 (+ 1402us) server_negotiation.cc:407] Connection header received
0808 11:43:47.014598 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.014603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.014694 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.014850 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:47.017102 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.018212 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.020546 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.020943 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.021094 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.025467 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:47.025472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:47.025476 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:47.025508 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:47.028727 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:47.029779 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:47.029785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:47.029790 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:47.029928 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:47.030352 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:47.030356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:47.030359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:47.030631 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:47.030809 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.031802 (+ 993us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.032066 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:43:47.033593 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.011604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39087 (local address 127.13.158.67:32933)
0808 11:43:47.012442 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.012482 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.014324 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.015127 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.015135 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.015958 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:47.016842 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.016878 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.018391 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.018398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.020383 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.020396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.021195 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.021203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.021448 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.022228 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:47.022254 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:47.025237 (+ 2983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:47.028882 (+ 3645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:47.028893 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:47.028916 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:47.029589 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:47.030040 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:47.030047 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:47.030054 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:47.030233 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:47.030790 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:47.030797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:47.031273 (+ 476us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.031656 (+ 383us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.032244 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":350,"threads_started":1}
I20260808 11:43:47.044739 15945 heartbeater.cc:503] Master 127.13.158.126:39087 was elected leader, sending a full tablet report...
I20260808 11:43:47.062954 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.046233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:58659 (local address 127.13.158.126:39087)
0808 11:43:47.046436 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.046444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.047270 (+ 826us) server_negotiation.cc:407] Connection header received
0808 11:43:47.047521 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.047525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.047582 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.047659 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:47.049145 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.049832 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.052258 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.052724 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.052869 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.055792 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:47.055800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:47.055808 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:47.055844 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:47.059501 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:47.060477 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:47.060484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:47.060491 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:47.060633 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:47.061104 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:47.061107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:47.061109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:47.061453 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:47.061627 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.062232 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.062474 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 11:43:47.063230 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.046288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39087 (local address 127.13.158.67:58659)
0808 11:43:47.047148 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.047173 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.047392 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.047729 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.047736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.048495 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:47.048980 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.048996 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.049999 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.050007 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.052085 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.052106 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.052949 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.052960 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.053210 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.054037 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:47.054068 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:47.055534 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:47.059689 (+ 4155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:47.059696 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:47.059699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:47.060323 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:47.060757 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:47.060764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:47.060770 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:47.060969 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:47.061586 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:47.061593 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:47.061851 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.062071 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.062538 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":479,"threads_started":1}
I20260808 11:43:47.073293 16157 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:41631
I20260808 11:43:47.073321 16367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:41631 every 8 connection(s)
I20260808 11:43:47.074450 16157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:43:47.076540 16368 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39087
I20260808 11:43:47.076821 16368 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:47.077454 16368 heartbeater.cc:511] Master 127.13.158.126:39087 requested a full tablet report, sending...
I20260808 11:43:47.078850 15650 ts_manager.cc:194] Registered new tserver with Master: 6fcdba6532c44372a7614af195bbc6cd (127.13.158.67:41631)
I20260808 11:43:47.079627 15650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:32933
I20260808 11:43:47.082589 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 16157
I20260808 11:43:47.082819 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:43:47.083202 13945 external_mini_cluster.cc:1428] Setting key 069e4e654eeae3b10f79ecba923e6edf
I20260808 11:43:47.085902 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:43:47 dist-test-slave-39l8 krb5kdc[15583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189427, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:43:47 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189427, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:47.134240 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.120784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.13.158.126:39087)
0808 11:43:47.121033 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.121041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.121193 (+ 152us) server_negotiation.cc:407] Connection header received
0808 11:43:47.121403 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.121407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.121469 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.121555 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:47.122994 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.123687 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.124826 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.125225 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.125326 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.129189 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:47.129195 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:47.129198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:47.129230 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:47.131561 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:47.132207 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:47.132213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:47.132216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:47.132359 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:47.132756 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:47.132761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:47.132765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:47.133048 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:47.133321 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.133646 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.133880 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 11:43:47.138441 15650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:43:47.145385 15650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:47.179888 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.159860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.13.158.67:41631)
0808 11:43:47.171535 (+ 11675us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.171543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.172370 (+ 827us) server_negotiation.cc:407] Connection header received
0808 11:43:47.172472 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.172477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.172635 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.172812 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.173420 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.174998 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.177136 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.178622 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.178789 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.179059 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.179374 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.179537 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11489,"thread_start_us":11326,"threads_started":1}
I20260808 11:43:47.179932 16392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.159140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:41631 (local address 127.0.0.1:42566)
0808 11:43:47.172154 (+ 13014us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.172177 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.172291 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.172804 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.172808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.172832 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.173215 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.173225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.175141 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.175148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.176979 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.176991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.178870 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.178878 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.179096 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.179156 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.179308 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.179513 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12767,"mutex_wait_us":54,"thread_start_us":12588,"threads_started":1}
I20260808 11:43:47.180716 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.157321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39157 (local address 127.0.0.1:33140)
0808 11:43:47.158509 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.158554 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.158732 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.171497 (+ 12765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.171509 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.171553 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.172168 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.172191 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.174200 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.174205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.177193 (+ 2988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.177208 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.179012 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.179027 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.179186 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.179211 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.179267 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.180343 (+ 1076us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":194,"thread_start_us":1002,"threads_started":1}
I20260808 11:43:47.181690 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.158181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.13.158.66:36753)
0808 11:43:47.169954 (+ 11773us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.169968 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.171428 (+ 1460us) server_negotiation.cc:407] Connection header received
0808 11:43:47.171542 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.171550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.171901 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.172133 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.172907 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.174304 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.178828 (+ 4524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.180317 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.180470 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.180678 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.181242 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.181411 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11532,"thread_start_us":11791,"threads_started":1}
I20260808 11:43:47.181691 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.157572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.13.158.65:39157)
0808 11:43:47.170675 (+ 13103us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.170684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.170739 (+ 55us) server_negotiation.cc:407] Connection header received
0808 11:43:47.170875 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.170887 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.171210 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.172020 (+ 810us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.172397 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.174048 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.177360 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.178753 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.180978 (+ 2225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.181186 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.181249 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.181401 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12916,"thread_start_us":12773,"threads_started":1}
I20260808 11:43:47.183938 16259 tablet_service.cc:1467] Processing CreateTablet for tablet f90f6d49a0c34983a09c47d7c235354c (DEFAULT_TABLE table=test-table [id=447df712b7cc4455874b03c836d2831c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:47.184788 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.157934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:36753 (local address 127.0.0.1:53586)
0808 11:43:47.171104 (+ 13170us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.171133 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.171284 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.172284 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.172290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.172319 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.172716 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.172728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.174475 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.174481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.177193 (+ 2712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.177208 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.180855 (+ 3647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.180862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.181023 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.181053 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.181153 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.181285 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13008,"mutex_wait_us":102,"thread_start_us":12872,"threads_started":1}
I20260808 11:43:47.184391 15851 tablet_service.cc:1467] Processing CreateTablet for tablet f90f6d49a0c34983a09c47d7c235354c (DEFAULT_TABLE table=test-table [id=447df712b7cc4455874b03c836d2831c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:47.185418 16259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90f6d49a0c34983a09c47d7c235354c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.185583 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90f6d49a0c34983a09c47d7c235354c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.188172 16048 tablet_service.cc:1467] Processing CreateTablet for tablet f90f6d49a0c34983a09c47d7c235354c (DEFAULT_TABLE table=test-table [id=447df712b7cc4455874b03c836d2831c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:47.189461 16048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90f6d49a0c34983a09c47d7c235354c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.197299 16394 tablet_bootstrap.cc:492] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Bootstrap starting.
I20260808 11:43:47.200189 16395 tablet_bootstrap.cc:492] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Bootstrap starting.
I20260808 11:43:47.200814 16394 tablet_bootstrap.cc:654] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.202162 16394 log.cc:824] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:47.203120 16397 tablet_bootstrap.cc:492] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Bootstrap starting.
I20260808 11:43:47.203986 16395 tablet_bootstrap.cc:654] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.205034 16395 log.cc:824] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:47.206357 16397 tablet_bootstrap.cc:654] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.206625 16394 tablet_bootstrap.cc:492] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: No bootstrap required, opened a new log
I20260808 11:43:47.206981 16394 ts_tablet_manager.cc:1397] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260808 11:43:47.207873 16397 log.cc:824] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:47.208211 16395 tablet_bootstrap.cc:492] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: No bootstrap required, opened a new log
I20260808 11:43:47.208505 16395 ts_tablet_manager.cc:1397] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 11:43:47.210888 16397 tablet_bootstrap.cc:492] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: No bootstrap required, opened a new log
I20260808 11:43:47.211141 16397 ts_tablet_manager.cc:1397] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260808 11:43:47.214859 16394 raft_consensus.cc:348] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.215222 16394 raft_consensus.cc:374] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.215354 16394 raft_consensus.cc:729] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d3947f42fd4982b8794c66cb154cf8, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.215965 16394 consensus_queue.cc:260] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.217900 16394 ts_tablet_manager.cc:1428] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260808 11:43:47.222348 16395 raft_consensus.cc:348] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.222827 16395 raft_consensus.cc:374] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.222937 16395 raft_consensus.cc:729] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fcdba6532c44372a7614af195bbc6cd, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.223628 16395 consensus_queue.cc:260] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.225466 16397 raft_consensus.cc:348] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.226042 16397 raft_consensus.cc:374] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.226155 16397 raft_consensus.cc:729] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634b716382f444fcad41dfc61960d950, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.226974 16397 consensus_queue.cc:260] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.231956 16152 heartbeater.cc:503] Master 127.13.158.126:39087 was elected leader, sending a full tablet report...
I20260808 11:43:47.232338 16397 ts_tablet_manager.cc:1428] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Time spent starting tablet: real 0.021s user 0.014s sys 0.002s
I20260808 11:43:47.232538 16395 ts_tablet_manager.cc:1428] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Time spent starting tablet: real 0.024s user 0.009s sys 0.012s
I20260808 11:43:47.232889 16368 heartbeater.cc:503] Master 127.13.158.126:39087 was elected leader, sending a full tablet report...
W20260808 11:43:47.287580 15946 tablet.cc:2369] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:43:47.300679 16153 tablet.cc:2369] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:43:47.326133 16371 tablet.cc:2369] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:43:47.371124 16402 raft_consensus.cc:482] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:47.371475 16402 raft_consensus.cc:504] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.373356 16402 leader_election.cc:288] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d3947f42fd4982b8794c66cb154cf8 (127.13.158.65:39157), 634b716382f444fcad41dfc61960d950 (127.13.158.66:36753)
I20260808 11:43:47.380039 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.373689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:38895 (local address 127.13.158.65:39157)
0808 11:43:47.373912 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.373918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.373941 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:43:47.374178 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.374187 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.374313 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.374421 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.375003 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.376023 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.378231 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.379213 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.379363 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.379417 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.379747 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.379837 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:43:47.380152 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.373605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39157 (local address 127.13.158.67:38895)
0808 11:43:47.373827 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.373853 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.373996 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.374529 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.374535 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.374569 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.374841 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.374857 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.376156 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.376159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.378059 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.378075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.379383 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.379389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.379573 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.379609 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.379711 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.379849 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 11:43:47.380959 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.374162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:38629 (local address 127.13.158.66:36753)
0808 11:43:47.374379 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.374387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.374896 (+ 509us) server_negotiation.cc:407] Connection header received
0808 11:43:47.375026 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.375030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.375109 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.375201 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.375765 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.376592 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.378884 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.379946 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.380115 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.380201 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.380566 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.380694 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:43:47.381013 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.373971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:36753 (local address 127.13.158.67:38629)
0808 11:43:47.374775 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.374793 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.374911 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.375221 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.375224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.375243 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.375608 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.375618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.376744 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.376749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.378669 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.378681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.380216 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.380223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.380385 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.380417 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.380518 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.380639 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":1,"thread_start_us":427,"threads_started":1}
I20260808 11:43:47.381142 15879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90f6d49a0c34983a09c47d7c235354c" candidate_uuid: "6fcdba6532c44372a7614af195bbc6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d3947f42fd4982b8794c66cb154cf8" is_pre_election: true
I20260808 11:43:47.381754 15879 raft_consensus.cc:2454] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fcdba6532c44372a7614af195bbc6cd in term 0.
I20260808 11:43:47.382038 16086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90f6d49a0c34983a09c47d7c235354c" candidate_uuid: "6fcdba6532c44372a7614af195bbc6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634b716382f444fcad41dfc61960d950" is_pre_election: true
I20260808 11:43:47.382436 16185 leader_election.cc:302] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fcdba6532c44372a7614af195bbc6cd, b1d3947f42fd4982b8794c66cb154cf8; no voters:
I20260808 11:43:47.382470 16086 raft_consensus.cc:2454] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fcdba6532c44372a7614af195bbc6cd in term 0.
I20260808 11:43:47.382822 16402 raft_consensus.cc:2781] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:47.382989 16402 raft_consensus.cc:482] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:47.383123 16402 raft_consensus.cc:3037] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.385290 16402 raft_consensus.cc:504] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.385963 16402 leader_election.cc:288] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [CANDIDATE]: Term 1 election: Requested vote from peers b1d3947f42fd4982b8794c66cb154cf8 (127.13.158.65:39157), 634b716382f444fcad41dfc61960d950 (127.13.158.66:36753)
I20260808 11:43:47.386296 15879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90f6d49a0c34983a09c47d7c235354c" candidate_uuid: "6fcdba6532c44372a7614af195bbc6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d3947f42fd4982b8794c66cb154cf8"
I20260808 11:43:47.386500 15879 raft_consensus.cc:3037] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.386432 16086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90f6d49a0c34983a09c47d7c235354c" candidate_uuid: "6fcdba6532c44372a7614af195bbc6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634b716382f444fcad41dfc61960d950"
I20260808 11:43:47.386678 16086 raft_consensus.cc:3037] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.388978 16086 raft_consensus.cc:2454] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fcdba6532c44372a7614af195bbc6cd in term 1.
I20260808 11:43:47.389035 15879 raft_consensus.cc:2454] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fcdba6532c44372a7614af195bbc6cd in term 1.
I20260808 11:43:47.389452 16186 leader_election.cc:302] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634b716382f444fcad41dfc61960d950, 6fcdba6532c44372a7614af195bbc6cd; no voters:
I20260808 11:43:47.389794 16402 raft_consensus.cc:2781] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:47.390168 16402 raft_consensus.cc:686] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 1 LEADER]: Becoming Leader. State: Replica: 6fcdba6532c44372a7614af195bbc6cd, State: Running, Role: LEADER
I20260808 11:43:47.390669 16402 consensus_queue.cc:237] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } }
I20260808 11:43:47.395537 15649 catalog_manager.cc:5495] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fcdba6532c44372a7614af195bbc6cd (127.13.158.67). New cstate: current_term: 1 leader_uuid: "6fcdba6532c44372a7614af195bbc6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } health_report { overall_health: UNKNOWN } } }
I20260808 11:43:47.471787 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.466748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.13.158.67:41631)
0808 11:43:47.466980 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.466987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.467015 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:43:47.467268 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.467272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.467347 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.467458 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:47.467793 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.468512 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.469782 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.470169 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.470259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.470685 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:47.471188 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:47.471305 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.471427 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.471489 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:43:47.479676 16086 raft_consensus.cc:1261] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Refusing update from remote peer 6fcdba6532c44372a7614af195bbc6cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:47.479708 15879 raft_consensus.cc:1261] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Refusing update from remote peer 6fcdba6532c44372a7614af195bbc6cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:47.480674 16405 consensus_queue.cc:1035] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:47.481065 16402 consensus_queue.cc:1035] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:47.509938 15649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:47.510227 15649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:47.513630 15649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4: Sending DeleteTablet for 3 replicas of tablet f90f6d49a0c34983a09c47d7c235354c
I20260808 11:43:47.514762 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet f90f6d49a0c34983a09c47d7c235354c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:47 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:33140
I20260808 11:43:47.514838 16048 tablet_service.cc:1514] Processing DeleteTablet for tablet f90f6d49a0c34983a09c47d7c235354c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:47 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53586
I20260808 11:43:47.514823 16259 tablet_service.cc:1514] Processing DeleteTablet for tablet f90f6d49a0c34983a09c47d7c235354c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:47 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:42566
I20260808 11:43:47.517946 15650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:43:47.518872 15650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:47.529409 16047 tablet_service.cc:1467] Processing CreateTablet for tablet f485a0579a444163879f494f7ccdfcd0 (DEFAULT_TABLE table=test-table [id=31949bbe664346faaf554c54c2a95399]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:47.530092 16047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f485a0579a444163879f494f7ccdfcd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.531224 16426 tablet_replica.cc:330] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: stopping tablet replica
I20260808 11:43:47.531723 16426 raft_consensus.cc:2229] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:47.532307 16426 raft_consensus.cc:2258] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:47.534593 16426 ts_tablet_manager.cc:1905] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:47.535566 15850 tablet_service.cc:1467] Processing CreateTablet for tablet f485a0579a444163879f494f7ccdfcd0 (DEFAULT_TABLE table=test-table [id=31949bbe664346faaf554c54c2a95399]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:47.536042 15850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f485a0579a444163879f494f7ccdfcd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.539096 16257 tablet_service.cc:1467] Processing CreateTablet for tablet f485a0579a444163879f494f7ccdfcd0 (DEFAULT_TABLE table=test-table [id=31949bbe664346faaf554c54c2a95399]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:47.539733 16257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f485a0579a444163879f494f7ccdfcd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.544097 16424 tablet_replica.cc:330] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: stopping tablet replica
I20260808 11:43:47.544391 16426 ts_tablet_manager.cc:1918] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:43:47.544577 16426 log.cc:1196] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/f90f6d49a0c34983a09c47d7c235354c
I20260808 11:43:47.544788 16424 raft_consensus.cc:2229] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:47.545203 16426 ts_tablet_manager.cc:1939] T f90f6d49a0c34983a09c47d7c235354c P b1d3947f42fd4982b8794c66cb154cf8: Deleting consensus metadata
I20260808 11:43:47.545414 16424 raft_consensus.cc:2258] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:47.547366 15628 catalog_manager.cc:4841] TS b1d3947f42fd4982b8794c66cb154cf8 (127.13.158.65:39157): tablet f90f6d49a0c34983a09c47d7c235354c (table test-table [id=447df712b7cc4455874b03c836d2831c]) successfully deleted
I20260808 11:43:47.548856 16394 tablet_bootstrap.cc:492] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: Bootstrap starting.
I20260808 11:43:47.550422 16394 tablet_bootstrap.cc:654] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.552847 16424 ts_tablet_manager.cc:1905] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:47.553499 16395 tablet_bootstrap.cc:492] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: Bootstrap starting.
I20260808 11:43:47.555292 16395 tablet_bootstrap.cc:654] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.559103 16425 tablet_replica.cc:330] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: stopping tablet replica
I20260808 11:43:47.559154 16397 tablet_bootstrap.cc:492] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: Bootstrap starting.
I20260808 11:43:47.559556 16425 raft_consensus.cc:2229] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:43:47.560220 16425 raft_consensus.cc:2258] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:47.560914 16397 tablet_bootstrap.cc:654] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.561800 16424 ts_tablet_manager.cc:1918] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:43:47.561990 16424 log.cc:1196] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/f90f6d49a0c34983a09c47d7c235354c
I20260808 11:43:47.562619 16424 ts_tablet_manager.cc:1939] T f90f6d49a0c34983a09c47d7c235354c P 634b716382f444fcad41dfc61960d950: Deleting consensus metadata
I20260808 11:43:47.563743 16394 tablet_bootstrap.cc:492] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: No bootstrap required, opened a new log
I20260808 11:43:47.563908 16394 ts_tablet_manager.cc:1397] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.003s
I20260808 11:43:47.564498 16425 ts_tablet_manager.cc:1905] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:47.564774 16394 raft_consensus.cc:348] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.565001 16394 raft_consensus.cc:374] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.565102 16394 raft_consensus.cc:729] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d3947f42fd4982b8794c66cb154cf8, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.565359 16394 consensus_queue.cc:260] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.565917 16394 ts_tablet_manager.cc:1428] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 11:43:47.566212 15629 catalog_manager.cc:4841] TS 634b716382f444fcad41dfc61960d950 (127.13.158.66:36753): tablet f90f6d49a0c34983a09c47d7c235354c (table test-table [id=447df712b7cc4455874b03c836d2831c]) successfully deleted
I20260808 11:43:47.567780 16397 tablet_bootstrap.cc:492] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: No bootstrap required, opened a new log
I20260808 11:43:47.567950 16397 ts_tablet_manager.cc:1397] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260808 11:43:47.568969 16397 raft_consensus.cc:348] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.569236 16397 raft_consensus.cc:374] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.569373 16397 raft_consensus.cc:729] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634b716382f444fcad41dfc61960d950, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.569761 16397 consensus_queue.cc:260] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.571103 16397 ts_tablet_manager.cc:1428] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 11:43:47.572142 16395 tablet_bootstrap.cc:492] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: No bootstrap required, opened a new log
I20260808 11:43:47.572333 16395 ts_tablet_manager.cc:1397] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260808 11:43:47.573235 16395 raft_consensus.cc:348] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.573478 16395 raft_consensus.cc:374] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.573589 16395 raft_consensus.cc:729] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fcdba6532c44372a7614af195bbc6cd, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.574775 16395 consensus_queue.cc:260] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.576287 16395 ts_tablet_manager.cc:1428] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 11:43:47.581135 16425 ts_tablet_manager.cc:1918] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:43:47.581349 16425 log.cc:1196] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/f90f6d49a0c34983a09c47d7c235354c
I20260808 11:43:47.581840 16425 ts_tablet_manager.cc:1939] T f90f6d49a0c34983a09c47d7c235354c P 6fcdba6532c44372a7614af195bbc6cd: Deleting consensus metadata
I20260808 11:43:47.583470 15628 catalog_manager.cc:4841] TS 6fcdba6532c44372a7614af195bbc6cd (127.13.158.67:41631): tablet f90f6d49a0c34983a09c47d7c235354c (table test-table [id=447df712b7cc4455874b03c836d2831c]) successfully deleted
I20260808 11:43:47.655355 16401 raft_consensus.cc:482] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:47.655650 16401 raft_consensus.cc:504] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.657406 16401 leader_election.cc:288] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 634b716382f444fcad41dfc61960d950 (127.13.158.66:36753), 6fcdba6532c44372a7614af195bbc6cd (127.13.158.67:41631)
I20260808 11:43:47.666966 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.658482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:38115 (local address 127.13.158.67:41631)
0808 11:43:47.658688 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.658695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.659393 (+ 698us) server_negotiation.cc:407] Connection header received
0808 11:43:47.659744 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.659751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.659833 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.659902 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.660858 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.662192 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.665014 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.665918 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.666095 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.666175 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.666476 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.666603 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 11:43:47.667275 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.657979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:41631 (local address 127.13.158.65:38115)
0808 11:43:47.659251 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.659278 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.659547 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.660142 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.660148 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.660177 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.660629 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.660645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.662409 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.662419 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.664801 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.664816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.666126 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.666132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.666316 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.666340 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.666447 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.666570 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1113,"mutex_wait_us":32,"thread_start_us":2778,"threads_started":1}
I20260808 11:43:47.667739 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.658031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:55253 (local address 127.13.158.66:36753)
0808 11:43:47.658262 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.658269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.659729 (+ 1460us) server_negotiation.cc:407] Connection header received
0808 11:43:47.660002 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.660007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.660082 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.660148 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.661195 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.662038 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.664989 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.666335 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.666519 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.666598 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.667269 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.667425 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 11:43:47.668165 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f485a0579a444163879f494f7ccdfcd0" candidate_uuid: "b1d3947f42fd4982b8794c66cb154cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fcdba6532c44372a7614af195bbc6cd" is_pre_election: true
I20260808 11:43:47.668541 16296 raft_consensus.cc:2454] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d3947f42fd4982b8794c66cb154cf8 in term 0.
I20260808 11:43:47.668618 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.658020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:36753 (local address 127.13.158.65:55253)
0808 11:43:47.659615 (+ 1595us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.659634 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.659835 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.660361 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.660368 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.660397 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.660904 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.660917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.662164 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.662172 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.664801 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.664816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.666810 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.666817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.667049 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.667088 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.668004 (+ 916us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.668175 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1460,"mutex_wait_us":48,"thread_start_us":893,"threads_started":1}
I20260808 11:43:47.669203 15775 leader_election.cc:302] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fcdba6532c44372a7614af195bbc6cd, b1d3947f42fd4982b8794c66cb154cf8; no voters:
I20260808 11:43:47.669354 16086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f485a0579a444163879f494f7ccdfcd0" candidate_uuid: "b1d3947f42fd4982b8794c66cb154cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634b716382f444fcad41dfc61960d950" is_pre_election: true
I20260808 11:43:47.669667 16401 raft_consensus.cc:2781] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:47.669651 16086 raft_consensus.cc:2454] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d3947f42fd4982b8794c66cb154cf8 in term 0.
I20260808 11:43:47.669795 16401 raft_consensus.cc:482] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:47.669888 16401 raft_consensus.cc:3037] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.672219 16401 raft_consensus.cc:504] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.672832 16401 leader_election.cc:288] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [CANDIDATE]: Term 1 election: Requested vote from peers 634b716382f444fcad41dfc61960d950 (127.13.158.66:36753), 6fcdba6532c44372a7614af195bbc6cd (127.13.158.67:41631)
I20260808 11:43:47.673291 16086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f485a0579a444163879f494f7ccdfcd0" candidate_uuid: "b1d3947f42fd4982b8794c66cb154cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634b716382f444fcad41dfc61960d950"
I20260808 11:43:47.673416 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f485a0579a444163879f494f7ccdfcd0" candidate_uuid: "b1d3947f42fd4982b8794c66cb154cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fcdba6532c44372a7614af195bbc6cd"
I20260808 11:43:47.673513 16086 raft_consensus.cc:3037] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.673627 16296 raft_consensus.cc:3037] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.676631 16086 raft_consensus.cc:2454] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d3947f42fd4982b8794c66cb154cf8 in term 1.
I20260808 11:43:47.676963 16296 raft_consensus.cc:2454] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d3947f42fd4982b8794c66cb154cf8 in term 1.
I20260808 11:43:47.677107 15776 leader_election.cc:302] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634b716382f444fcad41dfc61960d950, b1d3947f42fd4982b8794c66cb154cf8; no voters:
I20260808 11:43:47.677439 16401 raft_consensus.cc:2781] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:47.677994 16401 raft_consensus.cc:686] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 1 LEADER]: Becoming Leader. State: Replica: b1d3947f42fd4982b8794c66cb154cf8, State: Running, Role: LEADER
I20260808 11:43:47.678426 16401 consensus_queue.cc:237] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.682200 15649 catalog_manager.cc:5495] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1d3947f42fd4982b8794c66cb154cf8 (127.13.158.65). New cstate: current_term: 1 leader_uuid: "b1d3947f42fd4982b8794c66cb154cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } health_report { overall_health: UNKNOWN } } }
Aug 08 11:43:47 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189425, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:47.739387 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.725338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39087 (local address 127.0.0.1:58770)
0808 11:43:47.726072 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.726096 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.726259 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.726601 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.726605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.727320 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:47.727529 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.727539 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.728616 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.728624 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.729614 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.729627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.730382 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.730388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.730547 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.731288 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:47.731309 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:47.733710 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:47.736787 (+ 3077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:47.736800 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:47.736808 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:47.737211 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:47.737559 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:47.737565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:47.737569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:47.737721 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:47.738259 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:47.738268 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:47.738433 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.738607 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.738943 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":383,"threads_started":1}
I20260808 11:43:47.739464 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.725389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.13.158.126:39087)
0808 11:43:47.726351 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.726358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.726386 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:43:47.726422 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.726425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.726470 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.726576 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:47.727688 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.728484 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.729760 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.730199 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.730333 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.733900 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:47.733905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:47.733908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:47.733937 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:47.736644 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:47.737326 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:47.737331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:47.737335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:47.737453 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:47.737825 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:47.737831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:47.737834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:47.738120 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:47.738281 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.738735 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.739067 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":141,"server-negotiator.queue_time_us":840,"thread_start_us":694,"threads_started":1}
I20260808 11:43:47.744786 15649 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:58770:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 11:43:47.745877 15649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:47.753551 15851 tablet_service.cc:1467] Processing CreateTablet for tablet 82c4c360cfc44bb7a279e3d2e2884eaa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ecef734d2d9e45c7a12348454bde1260]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:43:47.753729 16257 tablet_service.cc:1467] Processing CreateTablet for tablet 82c4c360cfc44bb7a279e3d2e2884eaa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ecef734d2d9e45c7a12348454bde1260]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:43:47.754215 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c4c360cfc44bb7a279e3d2e2884eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.754305 16257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c4c360cfc44bb7a279e3d2e2884eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.760377 16394 tablet_bootstrap.cc:492] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: Bootstrap starting.
I20260808 11:43:47.761067 16047 tablet_service.cc:1467] Processing CreateTablet for tablet 82c4c360cfc44bb7a279e3d2e2884eaa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ecef734d2d9e45c7a12348454bde1260]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:43:47.761608 16395 tablet_bootstrap.cc:492] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: Bootstrap starting.
I20260808 11:43:47.761711 16047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c4c360cfc44bb7a279e3d2e2884eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:47.762300 16394 tablet_bootstrap.cc:654] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.763197 16395 tablet_bootstrap.cc:654] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.771922 16397 tablet_bootstrap.cc:492] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: Bootstrap starting.
I20260808 11:43:47.772653 16395 tablet_bootstrap.cc:492] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: No bootstrap required, opened a new log
I20260808 11:43:47.772835 16395 ts_tablet_manager.cc:1397] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260808 11:43:47.773586 16397 tablet_bootstrap.cc:654] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:47.773663 16395 raft_consensus.cc:348] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.773911 16395 raft_consensus.cc:374] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.773991 16395 raft_consensus.cc:729] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fcdba6532c44372a7614af195bbc6cd, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.774338 16395 consensus_queue.cc:260] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.775104 16395 ts_tablet_manager.cc:1428] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 11:43:47.774981 16402 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:47.775321 16402 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:47.777316 16394 tablet_bootstrap.cc:492] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: No bootstrap required, opened a new log
I20260808 11:43:47.777396 16397 tablet_bootstrap.cc:492] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: No bootstrap required, opened a new log
I20260808 11:43:47.777516 16394 ts_tablet_manager.cc:1397] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.008s
I20260808 11:43:47.777575 16397 ts_tablet_manager.cc:1397] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 11:43:47.778412 16394 raft_consensus.cc:348] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.778733 16394 raft_consensus.cc:374] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.778584 16397 raft_consensus.cc:348] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.778832 16394 raft_consensus.cc:729] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d3947f42fd4982b8794c66cb154cf8, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.778883 16397 raft_consensus.cc:374] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:47.778966 16397 raft_consensus.cc:729] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634b716382f444fcad41dfc61960d950, State: Initialized, Role: FOLLOWER
I20260808 11:43:47.779245 16397 consensus_queue.cc:260] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.779347 16394 consensus_queue.cc:260] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.780022 16394 ts_tablet_manager.cc:1428] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 11:43:47.779760 16403 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:47.780156 16403 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:47.780447 16401 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:47.780757 16401 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:47.781316 16397 ts_tablet_manager.cc:1428] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 11:43:47.947046 16403 raft_consensus.cc:482] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:47.947264 16403 raft_consensus.cc:504] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.948942 16403 leader_election.cc:288] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d3947f42fd4982b8794c66cb154cf8 (127.13.158.65:39157), 6fcdba6532c44372a7614af195bbc6cd (127.13.158.67:41631)
I20260808 11:43:47.960902 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.949496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:39785 (local address 127.13.158.65:39157)
0808 11:43:47.950395 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.950403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.951067 (+ 664us) server_negotiation.cc:407] Connection header received
0808 11:43:47.951336 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.951343 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.951674 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.951799 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.953217 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.955086 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.958302 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.959567 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.959790 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.959943 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.960263 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.960427 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":573,"threads_started":1}
I20260808 11:43:47.960902 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.949557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39157 (local address 127.13.158.66:39785)
0808 11:43:47.950945 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.950962 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.951173 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.951863 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.951868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.951910 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.952956 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.952971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.955235 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.955241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.958080 (+ 2839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.958098 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.959908 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.959925 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.960085 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.960109 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.960225 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.960355 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"thread_start_us":814,"threads_started":1}
I20260808 11:43:47.962239 15879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c4c360cfc44bb7a279e3d2e2884eaa" candidate_uuid: "634b716382f444fcad41dfc61960d950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d3947f42fd4982b8794c66cb154cf8" is_pre_election: true
I20260808 11:43:47.962682 15879 raft_consensus.cc:2454] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634b716382f444fcad41dfc61960d950 in term 0.
I20260808 11:43:47.963416 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.951393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:41631 (local address 127.13.158.66:34743)
0808 11:43:47.953026 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:47.953043 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:47.953256 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:47.953580 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:47.953584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:47.953606 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:47.954280 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.954290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.956120 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.956127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.959687 (+ 3560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:47.959702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.962027 (+ 2325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.962033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:47.962433 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.962469 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:43:47.962597 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 11:43:47.962755 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1453,"thread_start_us":1062,"threads_started":1}
I20260808 11:43:47.963322 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:47.951519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:34743 (local address 127.13.158.67:41631)
0808 11:43:47.951767 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:47.951775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:47.953114 (+ 1339us) server_negotiation.cc:407] Connection header received
0808 11:43:47.953403 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:47.953413 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:47.953471 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:47.953566 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:47.954531 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.955927 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.959946 (+ 4019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:47.961648 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:47.961919 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:47.962092 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:47.962636 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 11:43:47.962801 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:43:47.964103 15978 leader_election.cc:302] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634b716382f444fcad41dfc61960d950, b1d3947f42fd4982b8794c66cb154cf8; no voters:
I20260808 11:43:47.964653 16403 raft_consensus.cc:2781] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:47.964874 16403 raft_consensus.cc:482] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:47.965065 16403 raft_consensus.cc:3037] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.965904 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c4c360cfc44bb7a279e3d2e2884eaa" candidate_uuid: "634b716382f444fcad41dfc61960d950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fcdba6532c44372a7614af195bbc6cd" is_pre_election: true
I20260808 11:43:47.966252 16296 raft_consensus.cc:2454] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634b716382f444fcad41dfc61960d950 in term 0.
I20260808 11:43:47.968809 16403 raft_consensus.cc:504] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.969631 16403 leader_election.cc:288] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [CANDIDATE]: Term 1 election: Requested vote from peers b1d3947f42fd4982b8794c66cb154cf8 (127.13.158.65:39157), 6fcdba6532c44372a7614af195bbc6cd (127.13.158.67:41631)
I20260808 11:43:47.970259 15879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c4c360cfc44bb7a279e3d2e2884eaa" candidate_uuid: "634b716382f444fcad41dfc61960d950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d3947f42fd4982b8794c66cb154cf8"
I20260808 11:43:47.970301 16296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c4c360cfc44bb7a279e3d2e2884eaa" candidate_uuid: "634b716382f444fcad41dfc61960d950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fcdba6532c44372a7614af195bbc6cd"
I20260808 11:43:47.970579 15879 raft_consensus.cc:3037] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.970604 16296 raft_consensus.cc:3037] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:47.973654 15879 raft_consensus.cc:2454] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634b716382f444fcad41dfc61960d950 in term 1.
I20260808 11:43:47.974064 15978 leader_election.cc:302] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634b716382f444fcad41dfc61960d950, b1d3947f42fd4982b8794c66cb154cf8; no voters:
I20260808 11:43:47.974416 16403 raft_consensus.cc:2781] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:47.975224 16403 raft_consensus.cc:686] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [term 1 LEADER]: Becoming Leader. State: Replica: 634b716382f444fcad41dfc61960d950, State: Running, Role: LEADER
I20260808 11:43:47.975524 16296 raft_consensus.cc:2454] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634b716382f444fcad41dfc61960d950 in term 1.
I20260808 11:43:47.975874 16403 consensus_queue.cc:237] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } }
I20260808 11:43:47.978935 16452 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: TxnStatusTablet state changed. Reason: New leader 634b716382f444fcad41dfc61960d950. Latest consensus state: current_term: 1 leader_uuid: "634b716382f444fcad41dfc61960d950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:47.980108 16452 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:43:47.981472 16456 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 11:43:47.981817 16456 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 11:43:47.981678 15650 catalog_manager.cc:5495] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 reported cstate change: term changed from 0 to 1, leader changed from <none> to 634b716382f444fcad41dfc61960d950 (127.13.158.66). New cstate: current_term: 1 leader_uuid: "634b716382f444fcad41dfc61960d950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } health_report { overall_health: UNKNOWN } } }
I20260808 11:43:48.036067 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.025277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:36753 (local address 127.0.0.1:53590)
0808 11:43:48.025861 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:48.025881 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:48.026012 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:48.027212 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:48.027216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:48.027229 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:43:48.027511 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:48.027521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.029441 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.029447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:48.031647 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:48.031671 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.033191 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.033198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:48.033603 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.033678 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:43:48.035143 (+ 1465us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:43:48.035175 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:43:48.035295 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 11:43:48.035529 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394}
I20260808 11:43:48.037083 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.026388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.13.158.66:36753)
0808 11:43:48.026568 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.026574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.026591 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:43:48.026686 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.026690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.026857 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.026987 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:48.027682 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.029211 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.031891 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.032898 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.033030 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.034023 (+ 993us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:48.034921 (+ 898us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:48.035086 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.036453 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.036629 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:43:48.055585 15879 raft_consensus.cc:1261] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Refusing update from remote peer 634b716382f444fcad41dfc61960d950: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:48.055724 16296 raft_consensus.cc:1261] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd [term 1 FOLLOWER]: Refusing update from remote peer 634b716382f444fcad41dfc61960d950: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:48.056836 16452 consensus_queue.cc:1035] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:48.057835 16403 consensus_queue.cc:1035] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:48.095486 16401 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: TxnStatusTablet state changed. Reason: New leader 634b716382f444fcad41dfc61960d950. Latest consensus state: current_term: 1 leader_uuid: "634b716382f444fcad41dfc61960d950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:48.096027 16401 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:48.110083 16463 mvcc.cc:205] Tried to move back new op lower bound from 7316231897303318528 to 7316231896993906688. Current Snapshot: MvccSnapshot[applied={T|T < 7316231897303318528}]
I20260808 11:43:48.120278 16452 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "634b716382f444fcad41dfc61960d950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:48.120762 16452 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:43:48.126816 16434 consensus_queue.cc:1035] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:48.135888 16401 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "634b716382f444fcad41dfc61960d950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:48.136405 16401 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P b1d3947f42fd4982b8794c66cb154cf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:48.138103 16402 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: TxnStatusTablet state changed. Reason: New leader 634b716382f444fcad41dfc61960d950. Latest consensus state: current_term: 1 leader_uuid: "634b716382f444fcad41dfc61960d950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:48.139801 16402 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:48.142292 16403 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "634b716382f444fcad41dfc61960d950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:48.146101 16403 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P 634b716382f444fcad41dfc61960d950: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:43:48.164265 16401 consensus_queue.cc:1035] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:48.169625 16402 tablet_replica.cc:439] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "634b716382f444fcad41dfc61960d950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d3947f42fd4982b8794c66cb154cf8" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39157 } } peers { permanent_uuid: "634b716382f444fcad41dfc61960d950" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36753 } } peers { permanent_uuid: "6fcdba6532c44372a7614af195bbc6cd" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 41631 } } }
I20260808 11:43:48.169886 16402 tablet_replica.cc:442] T 82c4c360cfc44bb7a279e3d2e2884eaa P 6fcdba6532c44372a7614af195bbc6cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:48.177450 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.129185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33148 (local address 127.13.158.65:39157)
0808 11:43:48.129417 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.129424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.149589 (+ 20165us) server_negotiation.cc:407] Connection header received
0808 11:43:48.152042 (+ 2453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.152067 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.152272 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.152423 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:48.153864 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.155331 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.172271 (+ 16940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.173135 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.173262 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.174205 (+ 943us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:48.175048 (+ 843us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:48.175191 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.176894 (+ 1703us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.176994 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:43:48.200937 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.191674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:47117 (local address 127.13.158.66:36753)
0808 11:43:48.191873 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.191880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.192996 (+ 1116us) server_negotiation.cc:407] Connection header received
0808 11:43:48.193180 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.193185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.193280 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.193368 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:48.194272 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.195644 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.198497 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.199395 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.199500 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.200143 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:48.200271 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:48.200344 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.200567 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.200630 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 11:43:48.201346 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.191654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:36753 (local address 127.13.158.65:47117)
0808 11:43:48.192841 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:48.192873 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:48.193032 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:48.193407 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:48.193412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:48.193438 (+ 26us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:43:48.194058 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:48.194090 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.195806 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.195812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:48.198343 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:48.198359 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.199560 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.199568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:48.199829 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.199910 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:43:48.200387 (+ 477us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:43:48.200408 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 11:43:48.200554 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 11:43:48.200739 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":734,"threads_started":1}
I20260808 11:43:48.250581 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.240013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:50171 (local address 127.13.158.65:39157)
0808 11:43:48.240339 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.240348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.241935 (+ 1587us) server_negotiation.cc:407] Connection header received
0808 11:43:48.242159 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.242164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.242293 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.242426 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:48.243435 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.245051 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.247372 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.248554 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.248714 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.249642 (+ 928us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:48.249802 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:48.249903 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.250062 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.250150 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:43:48.250926 16487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.240396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39157 (local address 127.13.158.66:50171)
0808 11:43:48.241830 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:48.241847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:48.242018 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:48.242501 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:48.242505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:48.242517 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:43:48.243209 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:48.243220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.245369 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.245375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:48.247230 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:48.247243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.248955 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.248962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:48.249268 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.249363 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:43:48.249905 (+ 542us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:43:48.249919 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 11:43:48.250021 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 11:43:48.250178 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"thread_start_us":1038,"threads_started":1}
I20260808 11:43:48.297744 15650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:48.298041 15650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:48.302352 15650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebafcb85a60344918fae55632a84dbd4: Sending DeleteTablet for 3 replicas of tablet f485a0579a444163879f494f7ccdfcd0
I20260808 11:43:48.303689 16047 tablet_service.cc:1514] Processing DeleteTablet for tablet f485a0579a444163879f494f7ccdfcd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:48 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53586
I20260808 11:43:48.304354 16257 tablet_service.cc:1514] Processing DeleteTablet for tablet f485a0579a444163879f494f7ccdfcd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:48 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:42566
I20260808 11:43:48.305498 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet f485a0579a444163879f494f7ccdfcd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:48 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:33140
I20260808 11:43:48.320778 16490 tablet_replica.cc:330] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: stopping tablet replica
I20260808 11:43:48.321197 16490 raft_consensus.cc:2229] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:48.321877 16490 raft_consensus.cc:2258] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:48.323865 16490 ts_tablet_manager.cc:1905] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:48.332029 16492 tablet_replica.cc:330] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: stopping tablet replica
I20260808 11:43:48.332895 16491 tablet_replica.cc:330] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: stopping tablet replica
I20260808 11:43:48.333299 16492 raft_consensus.cc:2229] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:43:48.333755 16491 raft_consensus.cc:2229] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:48.334756 16492 raft_consensus.cc:2258] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:48.335186 16491 raft_consensus.cc:2258] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:48.336697 16491 ts_tablet_manager.cc:1905] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:48.338372 16490 ts_tablet_manager.cc:1918] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:48.338596 16490 log.cc:1196] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/f485a0579a444163879f494f7ccdfcd0
I20260808 11:43:48.339282 16490 ts_tablet_manager.cc:1939] T f485a0579a444163879f494f7ccdfcd0 P 634b716382f444fcad41dfc61960d950: Deleting consensus metadata
I20260808 11:43:48.339856 16492 ts_tablet_manager.cc:1905] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:48.340888 15629 catalog_manager.cc:4841] TS 634b716382f444fcad41dfc61960d950 (127.13.158.66:36753): tablet f485a0579a444163879f494f7ccdfcd0 (table test-table [id=31949bbe664346faaf554c54c2a95399]) successfully deleted
I20260808 11:43:48.346771 16492 ts_tablet_manager.cc:1918] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:48.348431 16491 ts_tablet_manager.cc:1918] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:48.348742 16491 log.cc:1196] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/f485a0579a444163879f494f7ccdfcd0
I20260808 11:43:48.349443 16491 ts_tablet_manager.cc:1939] T f485a0579a444163879f494f7ccdfcd0 P 6fcdba6532c44372a7614af195bbc6cd: Deleting consensus metadata
I20260808 11:43:48.350805 15628 catalog_manager.cc:4841] TS 6fcdba6532c44372a7614af195bbc6cd (127.13.158.67:41631): tablet f485a0579a444163879f494f7ccdfcd0 (table test-table [id=31949bbe664346faaf554c54c2a95399]) successfully deleted
I20260808 11:43:48.347086 16492 log.cc:1196] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/f485a0579a444163879f494f7ccdfcd0
Aug 08 11:43:48 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189427, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:43:48.352386 16492 ts_tablet_manager.cc:1939] T f485a0579a444163879f494f7ccdfcd0 P b1d3947f42fd4982b8794c66cb154cf8: Deleting consensus metadata
I20260808 11:43:48.353849 15628 catalog_manager.cc:4841] TS b1d3947f42fd4982b8794c66cb154cf8 (127.13.158.65:39157): tablet f485a0579a444163879f494f7ccdfcd0 (table test-table [id=31949bbe664346faaf554c54c2a95399]) successfully deleted
I20260808 11:43:48.366279 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.314239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.13.158.65:39157)
0808 11:43:48.314490 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.314500 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.330128 (+ 15628us) server_negotiation.cc:407] Connection header received
0808 11:43:48.330559 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.330576 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.330721 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.330824 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:48.334327 (+ 3503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.335942 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.339614 (+ 3672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.340846 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.340965 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.354879 (+ 13914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:48.354909 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:48.354939 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:48.354990 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:48.361253 (+ 6263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.362567 (+ 1314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.362576 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.362579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.362795 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.363490 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.363497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.363505 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.364498 (+ 993us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:48.364892 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.365458 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.365744 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 11:43:48.367465 15939 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33162
I20260808 11:43:48.411994 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.372034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58786 (local address 127.13.158.126:39087)
0808 11:43:48.384607 (+ 12573us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.384614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.385472 (+ 858us) server_negotiation.cc:407] Connection header received
0808 11:43:48.385727 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.385733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.385835 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.385964 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:48.389795 (+ 3831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.391667 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.394085 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.395025 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.395171 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.400554 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:48.400562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:48.400567 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:48.400635 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:48.406383 (+ 5748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.407524 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.407529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.407533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.407742 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.408332 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.408338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.408341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.408833 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:48.409136 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.410617 (+ 1481us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.410918 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12372,"thread_start_us":12568,"threads_started":1}
W20260808 11:43:48.413419 15650 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58786
I20260808 11:43:48.421707 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 15750
I20260808 11:43:48.464423 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.423580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.13.158.65:39157)
0808 11:43:48.423802 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.423808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.439529 (+ 15721us) server_negotiation.cc:407] Connection header received
0808 11:43:48.439814 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.439820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.439928 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.440107 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:48.442641 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.444534 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.447094 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.448265 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.448443 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.453434 (+ 4991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:48.453442 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:48.453448 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:48.453497 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:48.459514 (+ 6017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.460702 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.460709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.460713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.460956 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.461539 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.461545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.461560 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.462124 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:48.462534 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.463155 (+ 621us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.463585 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 11:43:48.465747 15939 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33174
W20260808 11:43:48.466537 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 15750: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:48.466867 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 15750
I20260808 11:43:48.495937 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 15950
Aug 08 11:43:48 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189427, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:43:48.514350 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.496771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53596 (local address 127.13.158.66:36753)
0808 11:43:48.496961 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.496968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.497073 (+ 105us) server_negotiation.cc:407] Connection header received
0808 11:43:48.497305 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.497310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.497372 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.497505 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:48.499165 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.500142 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.501555 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.502012 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.502163 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.506747 (+ 4584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:48.506766 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:48.506790 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:48.506857 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:48.511214 (+ 4357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.511893 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.511904 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.511908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.512151 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.512528 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.512533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.512537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.513029 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:48.513432 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.513615 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.513853 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 11:43:48.515132 16145 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53596
W20260808 11:43:48.515754 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 15950: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:48.515930 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 15950
I20260808 11:43:48.542932 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 16157
Aug 08 11:43:48 dist-test-slave-39l8 krb5kdc[15583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189427, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:43:48.558256 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:48.543748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.13.158.67:41631)
0808 11:43:48.544202 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:48.544207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:48.544225 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:43:48.544297 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:48.544301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:48.544366 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:48.544464 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:48.545673 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.546462 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.547697 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:48.548138 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:48.548258 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:48.552451 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:48.552466 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:48.552483 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:48.552522 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:48.555768 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.556329 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.556336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.556338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.556475 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:48.556864 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:48.556871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:48.556877 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:48.557257 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:48.557493 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:48.557700 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 11:43:48.557875 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":236,"threads_started":1}
W20260808 11:43:48.559003 16358 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42582
W20260808 11:43:48.559820 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 16157: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:48.560007 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 16157
I20260808 11:43:48.588569 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 15603
I20260808 11:43:48.589761 15705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:48.706173 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 15603
2026-08-08T11:43:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5211 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:48 dist-test-slave-39l8 krb5kdc[16510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:48 dist-test-slave-39l8 krb5kdc[16510](info): set up 2 sockets
Aug 08 11:43:48 dist-test-slave-39l8 krb5kdc[16510](info): commencing operation
krb5kdc: starting...
W20260808 11:43:49.972950 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:43:50 dist-test-slave-39l8 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:43:50.172254 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:34227
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:34227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:50.496572 16532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:50.496876 16532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:50.497063 16532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:50.505070 16532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:50.505174 16532 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 11:43:50.505255 16532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:50.505347 16532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:50.505422 16532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:50.505517 16532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:50.515290 16532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:34227
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:34227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:50.516616 16532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:50.517941 16532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:50.529425 16540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:50.529505 16541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:50.529462 16544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:50.530864 16532 server_base.cc:1034] running on GCE node
I20260808 11:43:50.531617 16532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:50.532898 16532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:50.534236 16532 hybrid_clock.cc:648] HybridClock initialized: now 1786189430534141 us; error 53 us; skew 500 ppm
I20260808 11:43:50.534499 16532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:50 dist-test-slave-39l8 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189430, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:50.537562 16532 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:43:50.538967 16532 webserver.cc:460] Webserver started at http://127.13.158.126:40071/ using document root <none> and password file <none>
I20260808 11:43:50.539623 16532 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:50.539709 16532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:50.539980 16532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:50.542899 16532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "5eed45be70b842d785d268b3c6ece825"
format_stamp: "Formatted at 2026-08-08 11:43:50 on dist-test-slave-39l8"
server_key: "a99d7ab1a184e663f93384ee03da2ce1"
server_key_iv: "5875ec2eb2cb6c7955552aa892df5616"
server_key_version: "encryptionkey@0"
I20260808 11:43:50.543602 16532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "5eed45be70b842d785d268b3c6ece825"
format_stamp: "Formatted at 2026-08-08 11:43:50 on dist-test-slave-39l8"
server_key: "a99d7ab1a184e663f93384ee03da2ce1"
server_key_iv: "5875ec2eb2cb6c7955552aa892df5616"
server_key_version: "encryptionkey@0"
I20260808 11:43:50.548307 16532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:43:50.551632 16555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:50.552909 16532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:43:50.553110 16532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "5eed45be70b842d785d268b3c6ece825"
format_stamp: "Formatted at 2026-08-08 11:43:50 on dist-test-slave-39l8"
server_key: "a99d7ab1a184e663f93384ee03da2ce1"
server_key_iv: "5875ec2eb2cb6c7955552aa892df5616"
server_key_version: "encryptionkey@0"
I20260808 11:43:50.553349 16532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:50.563985 16532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:50.568265 16532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:50.568564 16532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:50.590008 16532 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:34227
I20260808 11:43:50.590019 16646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:34227 every 8 connection(s)
I20260808 11:43:50.591507 16532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:50.595340 16650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:50.600777 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 16532
I20260808 11:43:50.601043 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:43:50.601686 13945 external_mini_cluster.cc:1428] Setting key 83b7509b8baecc49d319aec429f006cb
I20260808 11:43:50.606323 16650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: Bootstrap starting.
I20260808 11:43:50.610091 16650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:50.611394 16650 log.cc:824] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:50.614504 16650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: No bootstrap required, opened a new log
Aug 08 11:43:50 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:50.623090 16650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed45be70b842d785d268b3c6ece825" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34227 } }
I20260808 11:43:50.623454 16650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:50.623577 16650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eed45be70b842d785d268b3c6ece825, State: Initialized, Role: FOLLOWER
I20260808 11:43:50.624171 16650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed45be70b842d785d268b3c6ece825" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34227 } }
I20260808 11:43:50.624440 16650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:50.624650 16650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:50.624915 16650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:50.626871 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:50.603782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38918 (local address 127.13.158.126:34227)
0808 11:43:50.604603 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:50.604619 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:50.604665 (+ 46us) server_negotiation.cc:407] Connection header received
0808 11:43:50.606933 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:50.606960 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:50.607427 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:50.607841 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:50.609114 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:50.611261 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:50.612955 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:50.613943 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:50.614124 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:50.618826 (+ 4702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:50.618835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:50.618849 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:50.618889 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:50.621973 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:50.622679 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:50.622687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:50.622693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:50.622896 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:50.623333 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:50.623338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:50.623340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:50.624051 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:50.624361 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:50.624901 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 11:43:50.625501 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":341,"threads_started":1}
I20260808 11:43:50.628058 16650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed45be70b842d785d268b3c6ece825" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34227 } }
I20260808 11:43:50.628603 16650 leader_election.cc:302] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eed45be70b842d785d268b3c6ece825; no voters:
I20260808 11:43:50.629256 16650 leader_election.cc:288] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:50.629374 16659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:50.631461 16650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:50.631466 16659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [term 1 LEADER]: Becoming Leader. State: Replica: 5eed45be70b842d785d268b3c6ece825, State: Running, Role: LEADER
I20260808 11:43:50.632123 16659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed45be70b842d785d268b3c6ece825" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34227 } }
I20260808 11:43:50.635469 16660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eed45be70b842d785d268b3c6ece825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed45be70b842d785d268b3c6ece825" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34227 } } }
I20260808 11:43:50.635385 16661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eed45be70b842d785d268b3c6ece825. Latest consensus state: current_term: 1 leader_uuid: "5eed45be70b842d785d268b3c6ece825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed45be70b842d785d268b3c6ece825" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34227 } } }
I20260808 11:43:50.635842 16660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:50.635847 16661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:50.637372 16674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:50.642800 16674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:50.651748 16674 catalog_manager.cc:1347] Generated new cluster ID: a6fc6acce0244be3ac724ad421eea8d7
I20260808 11:43:50.651906 16674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:50.662967 16674 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:50.664220 16674 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:50.672077 16674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: Generated new TSK 0
I20260808 11:43:50.672820 16674 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:43:50.731866 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:34227
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:51.074414 16688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:51.074689 16688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:51.074910 16688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:51.081960 16688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:51.082084 16688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:51.082247 16688 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 11:43:51.082324 16688 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:43:51.082458 16688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:43:51.092083 16688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:34227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:51.093359 16688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:51.094671 16688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:51.108350 16700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:51.108443 16697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:51.108417 16694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:51.109534 16688 server_base.cc:1034] running on GCE node
I20260808 11:43:51.110110 16688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:51.111199 16688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:51.112453 16688 hybrid_clock.cc:648] HybridClock initialized: now 1786189431112402 us; error 59 us; skew 500 ppm
I20260808 11:43:51.112803 16688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:51 dist-test-slave-39l8 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189431, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:51.120743 16688 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:43:51.122217 16688 webserver.cc:460] Webserver started at http://127.13.158.65:46769/ using document root <none> and password file <none>
I20260808 11:43:51.122985 16688 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:51.123131 16688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:51.123446 16688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:51.126752 16688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "a79976a2ca0843d6bff7ac684dcaea1f"
format_stamp: "Formatted at 2026-08-08 11:43:51 on dist-test-slave-39l8"
server_key: "69356cba26ae912d803a27c1593d1f92"
server_key_iv: "e0118b449f4774dd41a62e0ac2b1e42b"
server_key_version: "encryptionkey@0"
I20260808 11:43:51.127538 16688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "a79976a2ca0843d6bff7ac684dcaea1f"
format_stamp: "Formatted at 2026-08-08 11:43:51 on dist-test-slave-39l8"
server_key: "69356cba26ae912d803a27c1593d1f92"
server_key_iv: "e0118b449f4774dd41a62e0ac2b1e42b"
server_key_version: "encryptionkey@0"
I20260808 11:43:51.132966 16688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 11:43:51.136561 16708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:51.137918 16688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:43:51.138062 16688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "a79976a2ca0843d6bff7ac684dcaea1f"
format_stamp: "Formatted at 2026-08-08 11:43:51 on dist-test-slave-39l8"
server_key: "69356cba26ae912d803a27c1593d1f92"
server_key_iv: "e0118b449f4774dd41a62e0ac2b1e42b"
server_key_version: "encryptionkey@0"
I20260808 11:43:51.138227 16688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:51.150672 16688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:51.154984 16688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:51.155220 16688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:51.157881 16688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:51.158006 16688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:51.158144 16688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:51.158238 16688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:43:51 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189431, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:51.193250 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:51.158581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:37475 (local address 127.13.158.126:34227)
0808 11:43:51.171232 (+ 12651us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:51.171241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:51.171274 (+ 33us) server_negotiation.cc:407] Connection header received
0808 11:43:51.171370 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:51.171378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:51.171475 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:51.172274 (+ 799us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:51.174366 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.175771 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.178335 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.178854 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.179005 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:51.184002 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:51.184009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:51.184014 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:51.184062 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:51.188139 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:51.189262 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:51.189270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:51.189276 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:51.189407 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:51.189972 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:51.189978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:51.189982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:51.190400 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:51.190565 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:51.192663 (+ 2098us) server_negotiation.cc:299] Negotiation successful
0808 11:43:51.192879 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12417,"thread_start_us":12300,"threads_started":1}
I20260808 11:43:51.193308 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:51.159296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:34227 (local address 127.13.158.65:37475)
0808 11:43:51.160394 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:51.160457 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:51.163536 (+ 3079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:51.172115 (+ 8579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:51.172131 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:51.173197 (+ 1066us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:51.174128 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:51.174150 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.175979 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.175983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:51.178158 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:51.178171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.179170 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.179180 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:51.179557 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:51.180433 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:51.180473 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:51.183751 (+ 3278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:51.188330 (+ 4579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:51.188341 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:51.188362 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:51.189069 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:51.189547 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:51.189551 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:51.189553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:51.189822 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:51.190594 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:51.190602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:51.191091 (+ 489us) client_negotiation.cc:769] Sending connection context
0808 11:43:51.191500 (+ 409us) client_negotiation.cc:240] Negotiation successful
0808 11:43:51.191923 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":452,"threads_started":1}
I20260808 11:43:51.210445 16688 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:39345
I20260808 11:43:51.210491 16884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:39345 every 8 connection(s)
I20260808 11:43:51.211828 16688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:43:51.213513 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 16688
I20260808 11:43:51.213747 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:43:51.214553 13945 external_mini_cluster.cc:1428] Setting key 431f46900c84bb07aa100deb731735b8
I20260808 11:43:51.214920 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:51.200326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:57997 (local address 127.13.158.126:34227)
0808 11:43:51.200550 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:51.200557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:51.201131 (+ 574us) server_negotiation.cc:407] Connection header received
0808 11:43:51.201278 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:51.201286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:51.201388 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:51.201532 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:51.202397 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.203674 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.205910 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.206334 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.206479 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:51.208994 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:51.208998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:51.209002 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:51.209032 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:51.211805 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:51.212740 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:51.212745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:51.212748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:51.212902 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:51.213337 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:51.213341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:51.213344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:51.213682 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:51.213850 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:51.214359 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 11:43:51.214563 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:43:51.214502 16885 heartbeater.cc:347] Connected to a master server at 127.13.158.126:34227
I20260808 11:43:51.214998 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:51.200353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:34227 (local address 127.13.158.65:57997)
0808 11:43:51.201035 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:51.201051 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:51.201160 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:51.201542 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:51.201545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:51.201965 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:51.202243 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:51.202252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.203921 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.203926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:51.205749 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:51.205761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.206524 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.206530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:51.206692 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:51.207488 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:51.207532 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:51.208806 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:51.211938 (+ 3132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:51.211944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:51.211947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:51.212568 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:51.213017 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:51.213021 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:51.213024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:51.213173 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:51.213916 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:51.213919 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:51.214113 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 11:43:51.214257 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 11:43:51.214584 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":372,"threads_started":1}
I20260808 11:43:51.215569 16885 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:51.216274 16885 heartbeater.cc:511] Master 127.13.158.126:34227 requested a full tablet report, sending...
I20260808 11:43:51.218273 16579 ts_manager.cc:194] Registered new tserver with Master: a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345)
I20260808 11:43:51.219848 16579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:37475
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:43:51.261335 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:34227
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:51.582520 16889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:51.582922 16889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:51.583117 16889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:51.590644 16889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:51.590828 16889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:51.590997 16889 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 11:43:51.591097 16889 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:43:51.591214 16889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:43:51.602072 16889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:34227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:51.603509 16889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:51.604887 16889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:51.618386 16896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:51.618413 16899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:51.618448 16897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:51.619508 16889 server_base.cc:1034] running on GCE node
I20260808 11:43:51.620092 16889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:51.620990 16889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:51.622188 16889 hybrid_clock.cc:648] HybridClock initialized: now 1786189431622123 us; error 56 us; skew 500 ppm
I20260808 11:43:51.622465 16889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:51 dist-test-slave-39l8 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189431, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:51.625597 16889 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:43:51.627262 16889 webserver.cc:460] Webserver started at http://127.13.158.66:42123/ using document root <none> and password file <none>
I20260808 11:43:51.627908 16889 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:51.628011 16889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:51.628530 16889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:51.632507 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "f0378a2245e94da7b0610b5ba8ec63f3"
format_stamp: "Formatted at 2026-08-08 11:43:51 on dist-test-slave-39l8"
server_key: "fbe371ddd02be11eeb993c58c7fe85ed"
server_key_iv: "88c2b149bfe8a661f34eab74a420fdea"
server_key_version: "encryptionkey@0"
I20260808 11:43:51.633354 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "f0378a2245e94da7b0610b5ba8ec63f3"
format_stamp: "Formatted at 2026-08-08 11:43:51 on dist-test-slave-39l8"
server_key: "fbe371ddd02be11eeb993c58c7fe85ed"
server_key_iv: "88c2b149bfe8a661f34eab74a420fdea"
server_key_version: "encryptionkey@0"
I20260808 11:43:51.638814 16889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:43:51.642475 16907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:51.643962 16889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:43:51.644173 16889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "f0378a2245e94da7b0610b5ba8ec63f3"
format_stamp: "Formatted at 2026-08-08 11:43:51 on dist-test-slave-39l8"
server_key: "fbe371ddd02be11eeb993c58c7fe85ed"
server_key_iv: "88c2b149bfe8a661f34eab74a420fdea"
server_key_version: "encryptionkey@0"
I20260808 11:43:51.644385 16889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:51.664633 16889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:51.668408 16889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:51.668723 16889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:51.671277 16889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:51.671423 16889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:51.671548 16889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:51.671643 16889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:43:51 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189431, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:51.695643 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:51.672020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:55671 (local address 127.13.158.126:34227)
0808 11:43:51.672257 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:51.672263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:51.673616 (+ 1353us) server_negotiation.cc:407] Connection header received
0808 11:43:51.676539 (+ 2923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:51.676544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:51.676642 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:51.676770 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:51.679443 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.680369 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.682873 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.683326 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.683455 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:51.687509 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:51.687517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:51.687524 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:51.687572 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:51.690994 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:51.692272 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:51.692280 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:51.692285 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:51.692418 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:51.693090 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:51.693095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:51.693099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:51.693488 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:51.693648 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:51.694816 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0808 11:43:51.695109 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 11:43:51.696451 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:51.672509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:34227 (local address 127.13.158.66:55671)
0808 11:43:51.673395 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:51.673462 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:51.676233 (+ 2771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:51.677049 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:51.677064 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:51.678092 (+ 1028us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:51.679209 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:51.679242 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.680559 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.680563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:51.682665 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:51.682680 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.683574 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.683582 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:51.683773 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:51.684499 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:51.684525 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:51.687245 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:51.691181 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:51.691191 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:51.691210 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:51.692105 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:51.692544 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:51.692549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:51.692553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:51.692907 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:51.693727 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:51.693760 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:51.694235 (+ 475us) client_negotiation.cc:769] Sending connection context
0808 11:43:51.694627 (+ 392us) client_negotiation.cc:240] Negotiation successful
0808 11:43:51.695169 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":336,"threads_started":1}
I20260808 11:43:51.720929 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:51.706354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:44703 (local address 127.13.158.126:34227)
0808 11:43:51.706606 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:51.706614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:51.707168 (+ 554us) server_negotiation.cc:407] Connection header received
0808 11:43:51.707394 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:51.707400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:51.707490 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:51.707610 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:51.709513 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.710474 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.711927 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.712380 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.712524 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:51.714404 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:51.714410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:51.714415 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:51.714452 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:51.717784 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:51.718652 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:51.718661 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:51.718664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:51.718841 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:51.719335 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:51.719341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:51.719344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:51.719702 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:51.719882 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:51.720338 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 11:43:51.720548 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 11:43:51.721058 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:51.706351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:34227 (local address 127.13.158.66:44703)
0808 11:43:51.707044 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:51.707079 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:51.707250 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:51.707841 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:51.707844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:51.708486 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:51.709285 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:51.709301 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.710698 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.710726 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:51.711770 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:51.711782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:51.712595 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:51.712604 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:51.712769 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:51.713412 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:51.713429 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:51.714218 (+ 789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:51.717891 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:51.717897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:51.717901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:51.718521 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:51.718961 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:51.718966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:51.718970 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:51.719163 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:51.719846 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:51.719851 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:51.720070 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 11:43:51.720251 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 11:43:51.720578 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":389,"threads_started":1}
I20260808 11:43:51.731603 16889 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:34457
I20260808 11:43:51.731640 17078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:34457 every 8 connection(s)
I20260808 11:43:51.732977 16889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:43:51.735209 17081 heartbeater.cc:347] Connected to a master server at 127.13.158.126:34227
I20260808 11:43:51.735488 17081 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:51.736239 17081 heartbeater.cc:511] Master 127.13.158.126:34227 requested a full tablet report, sending...
I20260808 11:43:51.737514 16579 ts_manager.cc:194] Registered new tserver with Master: f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:51.738308 16579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:55671
I20260808 11:43:51.741501 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 16889
I20260808 11:43:51.741647 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:43:51.742089 13945 external_mini_cluster.cc:1428] Setting key d1c95bf7fa01cb34c1b31672edd4afc7
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:43:51.785833 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:34227
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:52.112969 17085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:52.113294 17085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:52.113463 17085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:52.120518 17085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:52.120657 17085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:52.120819 17085 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 11:43:52.120908 17085 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:43:52.121049 17085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:43:52.130270 17085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:34227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:52.131611 17085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:52.133033 17085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:52.145874 17091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:52.145848 17092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:52.145896 17095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:52.146589 17085 server_base.cc:1034] running on GCE node
I20260808 11:43:52.147324 17085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:52.148250 17085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:52.149470 17085 hybrid_clock.cc:648] HybridClock initialized: now 1786189432149370 us; error 88 us; skew 500 ppm
I20260808 11:43:52.149735 17085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:52 dist-test-slave-39l8 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189432, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:52.152968 17085 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:43:52.154359 17085 webserver.cc:460] Webserver started at http://127.13.158.67:35391/ using document root <none> and password file <none>
I20260808 11:43:52.155027 17085 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:52.155128 17085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:52.155390 17085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:52.158385 17085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "d50bc29c8b00446a82f5e5082e8d6e37"
format_stamp: "Formatted at 2026-08-08 11:43:52 on dist-test-slave-39l8"
server_key: "d37ada03d01283b44f0cad38b1da4ca7"
server_key_iv: "561bac8698db360bfb183fa33ebe8840"
server_key_version: "encryptionkey@0"
I20260808 11:43:52.159178 17085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "d50bc29c8b00446a82f5e5082e8d6e37"
format_stamp: "Formatted at 2026-08-08 11:43:52 on dist-test-slave-39l8"
server_key: "d37ada03d01283b44f0cad38b1da4ca7"
server_key_iv: "561bac8698db360bfb183fa33ebe8840"
server_key_version: "encryptionkey@0"
I20260808 11:43:52.164275 17085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:43:52.167572 17104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:52.168834 17085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 11:43:52.168958 17085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "d50bc29c8b00446a82f5e5082e8d6e37"
format_stamp: "Formatted at 2026-08-08 11:43:52 on dist-test-slave-39l8"
server_key: "d37ada03d01283b44f0cad38b1da4ca7"
server_key_iv: "561bac8698db360bfb183fa33ebe8840"
server_key_version: "encryptionkey@0"
I20260808 11:43:52.169116 17085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:52.187021 17085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:52.190678 17085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:52.190927 17085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:52.193437 17085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:52.193562 17085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:52.193738 17085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:52.193837 17085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:43:52 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189432, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:52.218832 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.193865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:42397 (local address 127.13.158.126:34227)
0808 11:43:52.194114 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.194124 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.195626 (+ 1502us) server_negotiation.cc:407] Connection header received
0808 11:43:52.198795 (+ 3169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.198801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.198890 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.199030 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:52.201322 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.202687 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.205174 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.205959 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.206101 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.210560 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:52.210568 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:52.210576 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:52.210621 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:52.215111 (+ 4490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:52.215977 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:52.215985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:52.215992 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:52.216178 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:52.216686 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:52.216693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:52.216698 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:52.217141 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:52.217308 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.217983 (+ 675us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.218269 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:43:52.219449 17116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.194530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:34227 (local address 127.13.158.67:42397)
0808 11:43:52.195456 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.195509 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.198380 (+ 2871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.199319 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.199334 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.200206 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:52.201094 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.201118 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.202918 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.202927 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.204999 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.205017 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.206220 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.206238 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.206405 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.207179 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:52.207204 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:52.210240 (+ 3036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:52.215276 (+ 5036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:52.215283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:52.215299 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:52.215847 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:52.216324 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:52.216328 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:52.216332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:52.216572 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:52.217276 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:52.217282 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:52.217575 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.217785 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.218122 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":431,"threads_started":1}
I20260808 11:43:52.224300 16885 heartbeater.cc:503] Master 127.13.158.126:34227 was elected leader, sending a full tablet report...
I20260808 11:43:52.243592 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.228379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:38597 (local address 127.13.158.126:34227)
0808 11:43:52.228567 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.228574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.229137 (+ 563us) server_negotiation.cc:407] Connection header received
0808 11:43:52.229277 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.229284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.229396 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.229523 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:52.230363 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.231694 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.233719 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.234426 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.234582 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.236825 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:52.236832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:52.236836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:52.236873 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:52.240546 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:52.241518 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:52.241523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:52.241526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:52.241705 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:52.242100 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:52.242105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:52.242109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:52.242379 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:52.242566 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.243199 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.243354 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 11:43:52.244158 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.227885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:34227 (local address 127.13.158.67:38597)
0808 11:43:52.229008 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.229033 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.229173 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.229538 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.229541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.229933 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:52.230214 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.230222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.231894 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.231909 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.233537 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.233550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.234679 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.234690 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.234969 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.235634 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:52.235659 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:52.236548 (+ 889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:52.240694 (+ 4146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:52.240699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:52.240703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:52.241389 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:52.241851 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:52.241856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:52.241860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:52.241986 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:52.242556 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:52.242564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:52.242854 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.243084 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.243505 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"thread_start_us":804,"threads_started":1}
I20260808 11:43:52.256927 17085 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:45423
I20260808 11:43:52.256974 17259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:45423 every 8 connection(s)
I20260808 11:43:52.258189 17085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:43:52.260285 17260 heartbeater.cc:347] Connected to a master server at 127.13.158.126:34227
I20260808 11:43:52.260504 17260 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:52.261144 17260 heartbeater.cc:511] Master 127.13.158.126:34227 requested a full tablet report, sending...
I20260808 11:43:52.262422 16579 ts_manager.cc:194] Registered new tserver with Master: d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67:45423)
I20260808 11:43:52.263295 16579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:42397
I20260808 11:43:52.264973 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 17085
I20260808 11:43:52.265162 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:43:52.265760 13945 external_mini_cluster.cc:1428] Setting key f950f029fa38a99e652687129bf0668d
I20260808 11:43:52.268532 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:43:52 dist-test-slave-39l8 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:43:52 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:52.315989 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.303187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42294 (local address 127.13.158.126:34227)
0808 11:43:52.303427 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.303434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.303576 (+ 142us) server_negotiation.cc:407] Connection header received
0808 11:43:52.303722 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.303726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.303791 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.303905 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:52.305117 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.305855 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.307025 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.307459 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.307551 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.311215 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:52.311220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:52.311223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:52.311255 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:52.313851 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:52.314349 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:52.314352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:52.314355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:52.314468 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:52.314804 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:52.314807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:52.314809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:52.315076 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:52.315280 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.315465 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.315613 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 11:43:52.320021 16579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:43:52.327083 16579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:52.350474 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.338321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39345 (local address 127.0.0.1:52314)
0808 11:43:52.339338 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.339398 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.339595 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.343150 (+ 3555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.343159 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.343192 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:52.343656 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.343669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.345747 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.345753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.347781 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.347793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.349323 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.349331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.349585 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.349642 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.349776 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.349995 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":1,"thread_start_us":538,"threads_started":1}
I20260808 11:43:52.351368 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.338350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.13.158.65:39345)
0808 11:43:52.342439 (+ 4089us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.342446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.342473 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:43:52.342582 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.342589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.342884 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.343874 (+ 990us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:52.344076 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.345588 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.347926 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.349072 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.349225 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.349445 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.351048 (+ 1603us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.351164 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3949,"thread_start_us":3838,"threads_started":1}
I20260808 11:43:52.352671 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.340405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.13.158.67:45423)
0808 11:43:52.342033 (+ 1628us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.342044 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.344556 (+ 2512us) server_negotiation.cc:407] Connection header received
0808 11:43:52.344779 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.344786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.344968 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.345164 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:52.345721 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.347256 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.349765 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.351255 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.351352 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.351559 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.352149 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.352338 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1345,"spinlock_wait_cycles":37120,"thread_start_us":1109,"threads_started":1}
I20260808 11:43:52.352820 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.338398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:34457 (local address 127.0.0.1:35048)
0808 11:43:52.343179 (+ 4781us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.343206 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.343411 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.344382 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.344386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.344410 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:52.344826 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.344837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.347415 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.347419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.349963 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.349977 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.351958 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.351963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.352168 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.352197 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.352321 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.352443 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4666,"mutex_wait_us":22,"thread_start_us":4501,"threads_started":1}
I20260808 11:43:52.352828 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.338791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.13.158.66:34457)
0808 11:43:52.341322 (+ 2531us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.341333 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.343312 (+ 1979us) server_negotiation.cc:407] Connection header received
0808 11:43:52.343729 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.343737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.343953 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.344104 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:52.344998 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.347280 (+ 2282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.350152 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.351479 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.351593 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.351791 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.352378 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.352546 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2372,"thread_start_us":2233,"threads_started":1}
I20260808 11:43:52.353396 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.339687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:45423 (local address 127.0.0.1:53162)
0808 11:43:52.344437 (+ 4750us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.344458 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.344623 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.345154 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.345158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.345190 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:52.345508 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.345520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.347416 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.347419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.349597 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.349611 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.351724 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.351730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.351899 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.351941 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.353008 (+ 1067us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.353138 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4643,"mutex_wait_us":26,"spinlock_wait_cycles":3840,"thread_start_us":4519,"threads_started":1}
I20260808 11:43:52.354959 16783 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3d4a1a738341ab98703ec6bfaa74aa (DEFAULT_TABLE table=test-table [id=24b7c219cb3944d18e6d9a7514aad967]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:52.356431 16783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3d4a1a738341ab98703ec6bfaa74aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.356739 17175 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3d4a1a738341ab98703ec6bfaa74aa (DEFAULT_TABLE table=test-table [id=24b7c219cb3944d18e6d9a7514aad967]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:52.357487 16982 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3d4a1a738341ab98703ec6bfaa74aa (DEFAULT_TABLE table=test-table [id=24b7c219cb3944d18e6d9a7514aad967]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:52.358103 17175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3d4a1a738341ab98703ec6bfaa74aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.358625 16982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3d4a1a738341ab98703ec6bfaa74aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.368846 17287 tablet_bootstrap.cc:492] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Bootstrap starting.
I20260808 11:43:52.371342 17288 tablet_bootstrap.cc:492] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Bootstrap starting.
I20260808 11:43:52.372829 17287 tablet_bootstrap.cc:654] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.372962 17289 tablet_bootstrap.cc:492] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Bootstrap starting.
I20260808 11:43:52.374503 17287 log.cc:824] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:52.374821 17288 tablet_bootstrap.cc:654] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.375618 17289 tablet_bootstrap.cc:654] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.376154 17288 log.cc:824] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:52.376575 17289 log.cc:824] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:52.380370 17288 tablet_bootstrap.cc:492] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: No bootstrap required, opened a new log
I20260808 11:43:52.380757 17288 ts_tablet_manager.cc:1397] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260808 11:43:52.382822 17289 tablet_bootstrap.cc:492] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: No bootstrap required, opened a new log
I20260808 11:43:52.383106 17289 ts_tablet_manager.cc:1397] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.004s
I20260808 11:43:52.383489 17287 tablet_bootstrap.cc:492] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: No bootstrap required, opened a new log
I20260808 11:43:52.383756 17287 ts_tablet_manager.cc:1397] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260808 11:43:52.388345 17288 raft_consensus.cc:348] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.388693 17288 raft_consensus.cc:374] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.388823 17288 raft_consensus.cc:729] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79976a2ca0843d6bff7ac684dcaea1f, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.389405 17288 consensus_queue.cc:260] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.391027 17288 ts_tablet_manager.cc:1428] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260808 11:43:52.393428 17289 raft_consensus.cc:348] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.393828 17289 raft_consensus.cc:374] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.393929 17289 raft_consensus.cc:729] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.393970 17287 raft_consensus.cc:348] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.394418 17287 raft_consensus.cc:374] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.394531 17287 raft_consensus.cc:729] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0378a2245e94da7b0610b5ba8ec63f3, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.394555 17289 consensus_queue.cc:260] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.395151 17287 consensus_queue.cc:260] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.395803 17260 heartbeater.cc:503] Master 127.13.158.126:34227 was elected leader, sending a full tablet report...
I20260808 11:43:52.396526 17289 ts_tablet_manager.cc:1428] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260808 11:43:52.397465 17081 heartbeater.cc:503] Master 127.13.158.126:34227 was elected leader, sending a full tablet report...
I20260808 11:43:52.397940 17287 ts_tablet_manager.cc:1428] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260808 11:43:52.459594 17301 raft_consensus.cc:482] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:52.459959 17301 raft_consensus.cc:504] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.461601 17301 leader_election.cc:288] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67:45423)
W20260808 11:43:52.464427 16886 tablet.cc:2369] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:43:52.468603 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.462009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:47347 (local address 127.13.158.65:39345)
0808 11:43:52.462266 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.462275 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.462775 (+ 500us) server_negotiation.cc:407] Connection header received
0808 11:43:52.463043 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.463051 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.463150 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.463254 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:52.463807 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.464798 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.466654 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.467675 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.467867 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.467945 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.468229 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.468350 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:43:52.468730 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.461993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39345 (local address 127.13.158.66:47347)
0808 11:43:52.462624 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.462639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.462845 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.463295 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.463300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.463328 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:52.463655 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.463664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.464951 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.464957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.466505 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.466516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.467905 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.467911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.468064 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.468087 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.468203 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.468323 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":366,"threads_started":1}
I20260808 11:43:52.469205 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.463046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:57993 (local address 127.13.158.67:45423)
0808 11:43:52.463242 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.463248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.463459 (+ 211us) server_negotiation.cc:407] Connection header received
0808 11:43:52.463669 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.463681 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.463743 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.463839 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:52.464363 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.465288 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.467120 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.468114 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.468267 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.468341 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.468932 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.469038 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:43:52.469826 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.462793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:45423 (local address 127.13.158.66:57993)
0808 11:43:52.463360 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.463372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.463519 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.463853 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.463857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.463878 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:52.464193 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.464200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.465423 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.465430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.466968 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.466982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.468538 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.468544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.468746 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.468783 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.469412 (+ 629us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.469501 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":2,"thread_start_us":292,"threads_started":1}
I20260808 11:43:52.469861 16815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3d4a1a738341ab98703ec6bfaa74aa" candidate_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" is_pre_election: true
I20260808 11:43:52.470427 16815 raft_consensus.cc:2454] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0378a2245e94da7b0610b5ba8ec63f3 in term 0.
I20260808 11:43:52.470913 17204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3d4a1a738341ab98703ec6bfaa74aa" candidate_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" is_pre_election: true
I20260808 11:43:52.471192 16913 leader_election.cc:302] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a79976a2ca0843d6bff7ac684dcaea1f, f0378a2245e94da7b0610b5ba8ec63f3; no voters:
I20260808 11:43:52.471400 17204 raft_consensus.cc:2454] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0378a2245e94da7b0610b5ba8ec63f3 in term 0.
I20260808 11:43:52.471585 17301 raft_consensus.cc:2781] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:52.471773 17301 raft_consensus.cc:482] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:52.471906 17301 raft_consensus.cc:3037] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.474218 17301 raft_consensus.cc:504] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.474850 17301 leader_election.cc:288] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [CANDIDATE]: Term 1 election: Requested vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67:45423)
I20260808 11:43:52.475174 16815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3d4a1a738341ab98703ec6bfaa74aa" candidate_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f"
I20260808 11:43:52.475270 17204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3d4a1a738341ab98703ec6bfaa74aa" candidate_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50bc29c8b00446a82f5e5082e8d6e37"
I20260808 11:43:52.475369 16815 raft_consensus.cc:3037] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.475473 17204 raft_consensus.cc:3037] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.477396 16815 raft_consensus.cc:2454] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0378a2245e94da7b0610b5ba8ec63f3 in term 1.
I20260808 11:43:52.477579 17204 raft_consensus.cc:2454] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0378a2245e94da7b0610b5ba8ec63f3 in term 1.
I20260808 11:43:52.477813 16913 leader_election.cc:302] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a79976a2ca0843d6bff7ac684dcaea1f, f0378a2245e94da7b0610b5ba8ec63f3; no voters:
I20260808 11:43:52.478147 17301 raft_consensus.cc:2781] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:52.478575 17301 raft_consensus.cc:686] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 LEADER]: Becoming Leader. State: Replica: f0378a2245e94da7b0610b5ba8ec63f3, State: Running, Role: LEADER
I20260808 11:43:52.479043 17301 consensus_queue.cc:237] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.483776 16580 catalog_manager.cc:5495] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66). New cstate: current_term: 1 leader_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } health_report { overall_health: UNKNOWN } } }
W20260808 11:43:52.485698 17082 tablet.cc:2369] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:43:52.509786 17261 tablet.cc:2369] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 11:43:52 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189430, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:52.537611 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.524946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:34227 (local address 127.0.0.1:42302)
0808 11:43:52.525501 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.525516 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.525630 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.525920 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.525923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.526536 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:52.526856 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.526867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.527814 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.527819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.528787 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.528798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.529451 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.529455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.529569 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.530180 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:52.530197 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:52.532420 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:52.535452 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:52.535461 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:52.535466 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:52.535861 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:52.536165 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:52.536167 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:52.536170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:52.536306 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:52.536759 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:52.536765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:52.536906 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.537039 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.537245 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":299,"threads_started":1}
I20260808 11:43:52.537614 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.524897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.13.158.126:34227)
0808 11:43:52.525051 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.525056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.525600 (+ 544us) server_negotiation.cc:407] Connection header received
0808 11:43:52.525734 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.525737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.525795 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.525873 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:52.526979 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.527690 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.528905 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.529296 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.529386 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.532581 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:52.532586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:52.532589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:52.532618 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:52.535338 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:52.535952 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:52.535956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:52.535958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:52.536073 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:52.536394 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:52.536397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:52.536399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:52.536644 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:52.536761 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.537138 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.537375 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":56}
I20260808 11:43:52.542814 16580 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:42302:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 11:43:52.543833 16580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:52.550521 16982 tablet_service.cc:1467] Processing CreateTablet for tablet 6410601cc50a43018f3654a27c7dcfbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c0bcb92faf2c4d28a1c901b9b483d379]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:43:52.550397 16783 tablet_service.cc:1467] Processing CreateTablet for tablet 6410601cc50a43018f3654a27c7dcfbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c0bcb92faf2c4d28a1c901b9b483d379]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:43:52.550802 17175 tablet_service.cc:1467] Processing CreateTablet for tablet 6410601cc50a43018f3654a27c7dcfbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c0bcb92faf2c4d28a1c901b9b483d379]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:43:52.551381 16982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6410601cc50a43018f3654a27c7dcfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.551383 16783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6410601cc50a43018f3654a27c7dcfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.551638 17175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6410601cc50a43018f3654a27c7dcfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.557637 17289 tablet_bootstrap.cc:492] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: Bootstrap starting.
I20260808 11:43:52.558454 17288 tablet_bootstrap.cc:492] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: Bootstrap starting.
I20260808 11:43:52.559460 17289 tablet_bootstrap.cc:654] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.560036 17288 tablet_bootstrap.cc:654] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.561494 17287 tablet_bootstrap.cc:492] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: Bootstrap starting.
I20260808 11:43:52.561941 17289 tablet_bootstrap.cc:492] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: No bootstrap required, opened a new log
I20260808 11:43:52.562042 17289 ts_tablet_manager.cc:1397] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 11:43:52.562554 17288 tablet_bootstrap.cc:492] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: No bootstrap required, opened a new log
I20260808 11:43:52.562690 17288 ts_tablet_manager.cc:1397] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 11:43:52.562564 17289 raft_consensus.cc:348] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.562862 17289 raft_consensus.cc:374] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.563019 17289 raft_consensus.cc:729] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.563251 17287 tablet_bootstrap.cc:654] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.563275 17289 consensus_queue.cc:260] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.563900 17289 ts_tablet_manager.cc:1428] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:43:52.563807 17298 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.564204 17298 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:52.563493 17288 raft_consensus.cc:348] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.564627 17288 raft_consensus.cc:374] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.564771 17288 raft_consensus.cc:729] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79976a2ca0843d6bff7ac684dcaea1f, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.565027 17288 consensus_queue.cc:260] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.565713 17288 ts_tablet_manager.cc:1428] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 11:43:52.565604 17297 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.566036 17297 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:52.567284 17287 tablet_bootstrap.cc:492] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: No bootstrap required, opened a new log
I20260808 11:43:52.567473 17287 ts_tablet_manager.cc:1397] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 11:43:52.568233 17287 raft_consensus.cc:348] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.568450 17287 raft_consensus.cc:374] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.568616 17287 raft_consensus.cc:729] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0378a2245e94da7b0610b5ba8ec63f3, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.568850 17287 consensus_queue.cc:260] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.569197 17301 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.569376 17287 ts_tablet_manager.cc:1428] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 11:43:52.569509 17301 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:52.597889 17298 raft_consensus.cc:482] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:52.598099 17298 raft_consensus.cc:504] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.599771 17298 leader_election.cc:288] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:52.606843 17116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.600065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:34457 (local address 127.13.158.67:49853)
0808 11:43:52.600585 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.600606 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.600728 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.601320 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.601331 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.601377 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:52.601631 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.601646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.603078 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.603084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.604507 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.604525 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.606077 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.606085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.606290 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.606331 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.606428 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.606556 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392}
I20260808 11:43:52.606884 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.600305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:49853 (local address 127.13.158.66:34457)
0808 11:43:52.600502 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.600508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.600848 (+ 340us) server_negotiation.cc:407] Connection header received
0808 11:43:52.600927 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.600932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.601009 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.601099 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:52.601822 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.602946 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.604671 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.605917 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.606077 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.606148 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.606475 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.606627 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:43:52.607661 17018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6410601cc50a43018f3654a27c7dcfbf" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" is_pre_election: true
I20260808 11:43:52.607900 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.600655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:38561 (local address 127.13.158.65:39345)
0808 11:43:52.600847 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.600853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.601689 (+ 836us) server_negotiation.cc:407] Connection header received
0808 11:43:52.601977 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.601982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.602048 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.602106 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:43:52.602861 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.603980 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.605762 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.606903 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.607076 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.607155 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.607509 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.607631 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 11:43:52.607935 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.600123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39345 (local address 127.13.158.67:38561)
0808 11:43:52.601592 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.601610 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.601816 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.602308 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.602312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.602348 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:43:52.602683 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.602693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.604126 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.604131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.605596 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.605608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.607235 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.607240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.607351 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.607373 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.607473 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.607584 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"thread_start_us":1690,"threads_started":1}
I20260808 11:43:52.608079 17018 raft_consensus.cc:2454] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 0.
I20260808 11:43:52.608562 16815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6410601cc50a43018f3654a27c7dcfbf" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" is_pre_election: true
I20260808 11:43:52.608824 16815 raft_consensus.cc:2454] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 0.
I20260808 11:43:52.608788 17105 leader_election.cc:302] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d50bc29c8b00446a82f5e5082e8d6e37, f0378a2245e94da7b0610b5ba8ec63f3; no voters:
I20260808 11:43:52.609198 17298 raft_consensus.cc:2781] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:52.609421 17298 raft_consensus.cc:482] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:52.609525 17298 raft_consensus.cc:3037] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.611732 17298 raft_consensus.cc:504] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.612375 17298 leader_election.cc:288] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 election: Requested vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:52.612807 16815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6410601cc50a43018f3654a27c7dcfbf" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f"
I20260808 11:43:52.612860 17018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6410601cc50a43018f3654a27c7dcfbf" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378a2245e94da7b0610b5ba8ec63f3"
I20260808 11:43:52.613035 17018 raft_consensus.cc:3037] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.613057 16815 raft_consensus.cc:3037] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.616164 17018 raft_consensus.cc:2454] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 1.
I20260808 11:43:52.616146 16815 raft_consensus.cc:2454] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 1.
I20260808 11:43:52.616588 17107 leader_election.cc:302] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a79976a2ca0843d6bff7ac684dcaea1f, d50bc29c8b00446a82f5e5082e8d6e37; no voters:
I20260808 11:43:52.616897 17298 raft_consensus.cc:2781] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:52.617326 17298 raft_consensus.cc:686] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 LEADER]: Becoming Leader. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Running, Role: LEADER
I20260808 11:43:52.617825 17298 consensus_queue.cc:237] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.620987 17319 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: TxnStatusTablet state changed. Reason: New leader d50bc29c8b00446a82f5e5082e8d6e37. Latest consensus state: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.621390 17319 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:43:52.622182 17321 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 11:43:52.622442 17321 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 11:43:52.622326 16579 catalog_manager.cc:5495] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 reported cstate change: term changed from 0 to 1, leader changed from <none> to d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } health_report { overall_health: HEALTHY } } }
I20260808 11:43:52.645340 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.639169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.13.158.67:45423)
0808 11:43:52.639369 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.639376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.639564 (+ 188us) server_negotiation.cc:407] Connection header received
0808 11:43:52.639849 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.639852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.639934 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.640049 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:52.640477 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.641251 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.643211 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.643630 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.643759 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.644280 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:52.644775 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:52.644876 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.645036 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.645101 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:43:52.645462 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.639187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:45423 (local address 127.0.0.1:53174)
0808 11:43:52.639465 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.639487 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.639649 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.640118 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.640124 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.640140 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:43:52.640338 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.640351 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.641391 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.641397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.643045 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.643060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.643794 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.643800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.644026 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.644099 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:43:52.644890 (+ 791us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:43:52.644912 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.645008 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.645157 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 11:43:52.653605 16815 raft_consensus.cc:1261] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Refusing update from remote peer d50bc29c8b00446a82f5e5082e8d6e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:52.653581 17018 raft_consensus.cc:1261] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Refusing update from remote peer d50bc29c8b00446a82f5e5082e8d6e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:52.654482 17319 consensus_queue.cc:1035] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:52.654783 17298 consensus_queue.cc:1035] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:52.670544 17301 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: TxnStatusTablet state changed. Reason: New leader d50bc29c8b00446a82f5e5082e8d6e37. Latest consensus state: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.671002 17301 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:52.674136 17297 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: TxnStatusTablet state changed. Reason: New leader d50bc29c8b00446a82f5e5082e8d6e37. Latest consensus state: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.674434 17297 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:52.675166 17324 mvcc.cc:205] Tried to move back new op lower bound from 7316231916138938368 to 7316231916005982208. Current Snapshot: MvccSnapshot[applied={T|T < 7316231916138938368}]
I20260808 11:43:52.679477 17301 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.679764 17301 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P f0378a2245e94da7b0610b5ba8ec63f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:52.682018 17298 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.682439 17298 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:43:52.683106 17327 mvcc.cc:205] Tried to move back new op lower bound from 7316231916138938368 to 7316231916005982208. Current Snapshot: MvccSnapshot[applied={T|T < 7316231916138938368}]
I20260808 11:43:52.686094 17298 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.686432 17298 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P d50bc29c8b00446a82f5e5082e8d6e37: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:43:52.690922 17297 tablet_replica.cc:439] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } } }
I20260808 11:43:52.691197 17297 tablet_replica.cc:442] T 6410601cc50a43018f3654a27c7dcfbf P a79976a2ca0843d6bff7ac684dcaea1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:43:52.699858 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.693551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.13.158.66:34457)
0808 11:43:52.693747 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.693753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.693873 (+ 120us) server_negotiation.cc:407] Connection header received
0808 11:43:52.694091 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.694097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.694200 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.694360 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:52.694672 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.695849 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.697209 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.697841 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.698005 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.698401 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:52.699138 (+ 737us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:52.699301 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.699352 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.699436 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:43:52.716825 17344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.710650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:45423 (local address 127.13.158.66:39577)
0808 11:43:52.711395 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.711411 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.711536 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.711872 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.711875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.711884 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:43:52.712139 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.712145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.713527 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.713531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.714521 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.714533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.715603 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.715622 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.715831 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.715894 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:43:52.716369 (+ 475us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:43:52.716378 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.716444 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.716537 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":397,"threads_started":1}
I20260808 11:43:52.716877 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.710691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:39577 (local address 127.13.158.67:45423)
0808 11:43:52.710902 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.710909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.711509 (+ 600us) server_negotiation.cc:407] Connection header received
0808 11:43:52.711651 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.711657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.711755 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.711863 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:52.712281 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.713384 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.714666 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.715408 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.715538 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.716131 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:52.716256 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:52.716356 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.716495 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.716582 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 11:43:52.726130 16814 raft_consensus.cc:1261] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Refusing update from remote peer f0378a2245e94da7b0610b5ba8ec63f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:52.726284 17204 raft_consensus.cc:1261] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Refusing update from remote peer f0378a2245e94da7b0610b5ba8ec63f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:52.726945 17304 consensus_queue.cc:1035] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:52.727620 17301 consensus_queue.cc:1035] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:52.731709 17326 mvcc.cc:205] Tried to move back new op lower bound from 7316231916439564288 to 7316231915438182400. Current Snapshot: MvccSnapshot[applied={T|T < 7316231916439564288}]
I20260808 11:43:52.768719 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.760391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:45425 (local address 127.13.158.66:34457)
0808 11:43:52.760623 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:52.760632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:52.762728 (+ 2096us) server_negotiation.cc:407] Connection header received
0808 11:43:52.762949 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:52.762955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:52.763029 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:52.763119 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:52.763721 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.764853 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.766811 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.767251 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.767396 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.767895 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:52.768029 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:52.768123 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:52.768336 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 11:43:52.768420 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:43:52.769695 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:52.760360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:34457 (local address 127.13.158.67:45425)
0808 11:43:52.762603 (+ 2243us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:52.762620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:52.762816 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:52.763177 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:52.763183 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:52.763196 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:43:52.763524 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.763536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.765297 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.765301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.766618 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:52.766631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:52.767442 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:52.767452 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:52.767585 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:52.767650 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:43:52.768184 (+ 534us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:43:52.768195 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 11:43:52.769112 (+ 917us) client_negotiation.cc:240] Negotiation successful
0808 11:43:52.769254 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2059,"thread_start_us":1915,"threads_started":1}
I20260808 11:43:52.806613 16580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:52.806962 16580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:52.811190 16580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: Sending DeleteTablet for 3 replicas of tablet 8f3d4a1a738341ab98703ec6bfaa74aa
I20260808 11:43:52.812630 16783 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3d4a1a738341ab98703ec6bfaa74aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:52 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:52314
I20260808 11:43:52.813629 17355 tablet_replica.cc:330] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: stopping tablet replica
I20260808 11:43:52.814108 17355 raft_consensus.cc:2229] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:52.814518 17355 raft_consensus.cc:2258] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:52.814432 16580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:43:52.815079 16580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:52.816406 16982 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3d4a1a738341ab98703ec6bfaa74aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:52 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:35048
I20260808 11:43:52.816634 17355 ts_tablet_manager.cc:1905] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:52.819670 17175 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3d4a1a738341ab98703ec6bfaa74aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:52 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53162
I20260808 11:43:52.821098 17356 tablet_replica.cc:330] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: stopping tablet replica
I20260808 11:43:52.821548 17356 raft_consensus.cc:2229] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:43:52.822889 17356 raft_consensus.cc:2258] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:52.823843 17355 ts_tablet_manager.cc:1918] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:52.824056 17355 log.cc:1196] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/8f3d4a1a738341ab98703ec6bfaa74aa
I20260808 11:43:52.824700 17355 ts_tablet_manager.cc:1939] T 8f3d4a1a738341ab98703ec6bfaa74aa P a79976a2ca0843d6bff7ac684dcaea1f: Deleting consensus metadata
I20260808 11:43:52.824774 17357 tablet_replica.cc:330] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: stopping tablet replica
I20260808 11:43:52.826834 16558 catalog_manager.cc:4841] TS a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345): tablet 8f3d4a1a738341ab98703ec6bfaa74aa (table test-table [id=24b7c219cb3944d18e6d9a7514aad967]) successfully deleted
I20260808 11:43:52.827376 16783 tablet_service.cc:1467] Processing CreateTablet for tablet c60f7ed23d794300b69547528c37a4f6 (DEFAULT_TABLE table=test-table [id=8e471919ba83419e828415e8dbcdd644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:52.827788 16783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60f7ed23d794300b69547528c37a4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.826858 16980 tablet_service.cc:1467] Processing CreateTablet for tablet c60f7ed23d794300b69547528c37a4f6 (DEFAULT_TABLE table=test-table [id=8e471919ba83419e828415e8dbcdd644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:52.831148 16980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60f7ed23d794300b69547528c37a4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.825780 17170 tablet_service.cc:1467] Processing CreateTablet for tablet c60f7ed23d794300b69547528c37a4f6 (DEFAULT_TABLE table=test-table [id=8e471919ba83419e828415e8dbcdd644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:52.832018 17170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60f7ed23d794300b69547528c37a4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:52.832738 17356 ts_tablet_manager.cc:1905] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:52.836777 17288 tablet_bootstrap.cc:492] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: Bootstrap starting.
I20260808 11:43:52.838192 17287 tablet_bootstrap.cc:492] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: Bootstrap starting.
I20260808 11:43:52.838433 17289 tablet_bootstrap.cc:492] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: Bootstrap starting.
I20260808 11:43:52.838464 17288 tablet_bootstrap.cc:654] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.840045 17289 tablet_bootstrap.cc:654] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.840068 17287 tablet_bootstrap.cc:654] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:52.840548 17356 ts_tablet_manager.cc:1918] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:52.840737 17356 log.cc:1196] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/8f3d4a1a738341ab98703ec6bfaa74aa
I20260808 11:43:52.845322 17357 raft_consensus.cc:2229] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:52.846359 17357 raft_consensus.cc:2258] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:52.847589 17356 ts_tablet_manager.cc:1939] T 8f3d4a1a738341ab98703ec6bfaa74aa P f0378a2245e94da7b0610b5ba8ec63f3: Deleting consensus metadata
I20260808 11:43:52.849064 17357 ts_tablet_manager.cc:1905] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:52.849630 16556 catalog_manager.cc:4841] TS f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457): tablet 8f3d4a1a738341ab98703ec6bfaa74aa (table test-table [id=24b7c219cb3944d18e6d9a7514aad967]) successfully deleted
I20260808 11:43:52.850402 17289 tablet_bootstrap.cc:492] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: No bootstrap required, opened a new log
I20260808 11:43:52.850621 17289 ts_tablet_manager.cc:1397] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260808 11:43:52.851418 17289 raft_consensus.cc:348] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.851758 17289 raft_consensus.cc:374] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.851884 17289 raft_consensus.cc:729] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.852210 17289 consensus_queue.cc:260] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.852946 17289 ts_tablet_manager.cc:1428] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:43:52.853797 17287 tablet_bootstrap.cc:492] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: No bootstrap required, opened a new log
I20260808 11:43:52.854029 17287 ts_tablet_manager.cc:1397] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
I20260808 11:43:52.854907 17287 raft_consensus.cc:348] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.855283 17287 raft_consensus.cc:374] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.855415 17287 raft_consensus.cc:729] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0378a2245e94da7b0610b5ba8ec63f3, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.855719 17287 consensus_queue.cc:260] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.856350 17287 ts_tablet_manager.cc:1428] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:43:52.859898 17288 tablet_bootstrap.cc:492] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: No bootstrap required, opened a new log
I20260808 11:43:52.860071 17288 ts_tablet_manager.cc:1397] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.001s
I20260808 11:43:52.861095 17288 raft_consensus.cc:348] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.861347 17288 raft_consensus.cc:374] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:52.861428 17288 raft_consensus.cc:729] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79976a2ca0843d6bff7ac684dcaea1f, State: Initialized, Role: FOLLOWER
I20260808 11:43:52.861646 17288 consensus_queue.cc:260] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.862226 17288 ts_tablet_manager.cc:1428] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 11:43:52.863229 17357 ts_tablet_manager.cc:1918] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:52.863447 17357 log.cc:1196] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/8f3d4a1a738341ab98703ec6bfaa74aa
I20260808 11:43:52.864091 17357 ts_tablet_manager.cc:1939] T 8f3d4a1a738341ab98703ec6bfaa74aa P d50bc29c8b00446a82f5e5082e8d6e37: Deleting consensus metadata
I20260808 11:43:52.865783 16558 catalog_manager.cc:4841] TS d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67:45423): tablet 8f3d4a1a738341ab98703ec6bfaa74aa (table test-table [id=24b7c219cb3944d18e6d9a7514aad967]) successfully deleted
I20260808 11:43:52.970250 17298 raft_consensus.cc:482] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:52.970476 17298 raft_consensus.cc:504] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.971339 17298 leader_election.cc:288] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:52.971992 17018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60f7ed23d794300b69547528c37a4f6" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" is_pre_election: true
I20260808 11:43:52.972052 16815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60f7ed23d794300b69547528c37a4f6" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" is_pre_election: true
I20260808 11:43:52.972275 17018 raft_consensus.cc:2454] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 0.
I20260808 11:43:52.972347 16815 raft_consensus.cc:2454] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 0.
I20260808 11:43:52.972864 17107 leader_election.cc:302] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a79976a2ca0843d6bff7ac684dcaea1f, d50bc29c8b00446a82f5e5082e8d6e37; no voters:
I20260808 11:43:52.973192 17298 raft_consensus.cc:2781] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:52.973336 17298 raft_consensus.cc:482] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:52.973440 17298 raft_consensus.cc:3037] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.976891 17298 raft_consensus.cc:504] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.977972 17298 leader_election.cc:288] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 election: Requested vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:52.977979 16815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60f7ed23d794300b69547528c37a4f6" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f"
I20260808 11:43:52.978204 16815 raft_consensus.cc:3037] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.978406 17018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60f7ed23d794300b69547528c37a4f6" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378a2245e94da7b0610b5ba8ec63f3"
I20260808 11:43:52.978643 17018 raft_consensus.cc:3037] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:52.981223 16815 raft_consensus.cc:2454] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 1.
I20260808 11:43:52.981796 17107 leader_election.cc:302] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a79976a2ca0843d6bff7ac684dcaea1f, d50bc29c8b00446a82f5e5082e8d6e37; no voters:
I20260808 11:43:52.981925 17018 raft_consensus.cc:2454] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 1.
I20260808 11:43:52.982244 17298 raft_consensus.cc:2781] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:52.982543 17298 raft_consensus.cc:686] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 LEADER]: Becoming Leader. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Running, Role: LEADER
I20260808 11:43:52.982937 17298 consensus_queue.cc:237] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:52.986038 16580 catalog_manager.cc:5495] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 reported cstate change: term changed from 0 to 1, leader changed from <none> to d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } health_report { overall_health: HEALTHY } } }
I20260808 11:43:53.012483 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:53.005421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.13.158.67:45423)
0808 11:43:53.005587 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:53.005592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:53.005656 (+ 64us) server_negotiation.cc:407] Connection header received
0808 11:43:53.005953 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:53.005959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:53.006042 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:53.006162 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:53.006795 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:53.008047 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:53.010129 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:53.010916 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:53.011042 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:53.011516 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:53.011709 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:53.011797 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:53.012041 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 11:43:53.012130 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:43:53.029456 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:53.022885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:45423 (local address 127.13.158.67:33775)
0808 11:43:53.023176 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:53.023196 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:53.023328 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:53.023705 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:53.023710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:53.023724 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:43:53.023952 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:53.023967 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:53.025384 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:53.025390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:53.027021 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:53.027039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:53.028018 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:53.028026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:53.028325 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:53.028379 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:43:53.028833 (+ 454us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:43:53.028845 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 11:43:53.028929 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 11:43:53.029073 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":51}
I20260808 11:43:53.030181 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:53.022847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:33775 (local address 127.13.158.67:45423)
0808 11:43:53.023044 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:53.023050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:53.023286 (+ 236us) server_negotiation.cc:407] Connection header received
0808 11:43:53.023466 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:53.023472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:53.023560 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:53.023660 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:53.024106 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:53.025221 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:53.027183 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:53.027837 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:53.028008 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:53.028568 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:53.028710 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:53.028816 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:53.029759 (+ 943us) server_negotiation.cc:299] Negotiation successful
0808 11:43:53.029843 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":59}
I20260808 11:43:53.047366 17018 raft_consensus.cc:1261] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Refusing update from remote peer d50bc29c8b00446a82f5e5082e8d6e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:53.049185 17298 consensus_queue.cc:1035] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:53.050633 16815 raft_consensus.cc:1261] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Refusing update from remote peer d50bc29c8b00446a82f5e5082e8d6e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:53.051367 17366 consensus_queue.cc:1035] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:43:53.090583 17324 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 11:43:54.696691 16579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:54.697026 16579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:54.702155 16579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: Sending DeleteTablet for 3 replicas of tablet c60f7ed23d794300b69547528c37a4f6
I20260808 11:43:54.703126 16783 tablet_service.cc:1514] Processing DeleteTablet for tablet c60f7ed23d794300b69547528c37a4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:54 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:52314
I20260808 11:43:54.703280 16980 tablet_service.cc:1514] Processing DeleteTablet for tablet c60f7ed23d794300b69547528c37a4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:54 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:35048
I20260808 11:43:54.703416 17170 tablet_service.cc:1514] Processing DeleteTablet for tablet c60f7ed23d794300b69547528c37a4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:54 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53162
I20260808 11:43:54.714404 17396 tablet_replica.cc:330] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: stopping tablet replica
I20260808 11:43:54.714957 17396 raft_consensus.cc:2229] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:54.715621 17396 raft_consensus.cc:2258] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:54.717182 17396 ts_tablet_manager.cc:1905] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:54.719079 17398 tablet_replica.cc:330] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: stopping tablet replica
I20260808 11:43:54.719436 17398 raft_consensus.cc:2229] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:43:54.720624 17398 raft_consensus.cc:2258] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:54.722021 17398 ts_tablet_manager.cc:1905] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:54.722774 17397 tablet_replica.cc:330] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: stopping tablet replica
I20260808 11:43:54.723078 17397 raft_consensus.cc:2229] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:54.723573 17397 raft_consensus.cc:2258] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:54.724790 17397 ts_tablet_manager.cc:1905] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:54.728503 17396 ts_tablet_manager.cc:1918] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:43:54.728700 17396 log.cc:1196] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/c60f7ed23d794300b69547528c37a4f6
I20260808 11:43:54.729351 17396 ts_tablet_manager.cc:1939] T c60f7ed23d794300b69547528c37a4f6 P a79976a2ca0843d6bff7ac684dcaea1f: Deleting consensus metadata
I20260808 11:43:54.730759 16558 catalog_manager.cc:4841] TS a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345): tablet c60f7ed23d794300b69547528c37a4f6 (table test-table [id=8e471919ba83419e828415e8dbcdd644]) successfully deleted
I20260808 11:43:54.731652 17398 ts_tablet_manager.cc:1918] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:43:54.731918 17398 log.cc:1196] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/c60f7ed23d794300b69547528c37a4f6
I20260808 11:43:54.732873 17398 ts_tablet_manager.cc:1939] T c60f7ed23d794300b69547528c37a4f6 P d50bc29c8b00446a82f5e5082e8d6e37: Deleting consensus metadata
I20260808 11:43:54.734266 16558 catalog_manager.cc:4841] TS d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67:45423): tablet c60f7ed23d794300b69547528c37a4f6 (table test-table [id=8e471919ba83419e828415e8dbcdd644]) successfully deleted
I20260808 11:43:54.734763 17397 ts_tablet_manager.cc:1918] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:43:54.735059 17397 log.cc:1196] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/c60f7ed23d794300b69547528c37a4f6
I20260808 11:43:54.735494 17397 ts_tablet_manager.cc:1939] T c60f7ed23d794300b69547528c37a4f6 P f0378a2245e94da7b0610b5ba8ec63f3: Deleting consensus metadata
I20260808 11:43:54.736694 16556 catalog_manager.cc:4841] TS f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457): tablet c60f7ed23d794300b69547528c37a4f6 (table test-table [id=8e471919ba83419e828415e8dbcdd644]) successfully deleted
Aug 08 11:43:54 dist-test-slave-39l8 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189434, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:43:54 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189434, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:54.774319 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:54.756999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.13.158.126:34227)
0808 11:43:54.758013 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:54.758021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:54.758067 (+ 46us) server_negotiation.cc:407] Connection header received
0808 11:43:54.758137 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:54.758140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:54.758210 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:54.758303 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:54.759708 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:54.760703 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:54.762117 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:54.762670 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:54.762851 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:54.767963 (+ 5112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:54.767969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:54.767972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:54.768003 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:54.771630 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:54.772474 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:54.772478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:54.772480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:54.772657 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:54.773027 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:54.773031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:54.773034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:54.773384 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:54.773697 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:54.773817 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 11:43:54.773995 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":566,"threads_started":1}
I20260808 11:43:54.778316 16579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:43:54.779091 16579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:54.785910 16783 tablet_service.cc:1467] Processing CreateTablet for tablet 2ade6d762bf74ac0a3550a3943ebd322 (DEFAULT_TABLE table=test-table [id=6ad3c56c379c41218ddecc147d2b52be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:54.786410 16783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ade6d762bf74ac0a3550a3943ebd322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:54.786310 16980 tablet_service.cc:1467] Processing CreateTablet for tablet 2ade6d762bf74ac0a3550a3943ebd322 (DEFAULT_TABLE table=test-table [id=6ad3c56c379c41218ddecc147d2b52be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:54.786386 17170 tablet_service.cc:1467] Processing CreateTablet for tablet 2ade6d762bf74ac0a3550a3943ebd322 (DEFAULT_TABLE table=test-table [id=6ad3c56c379c41218ddecc147d2b52be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:54.786835 16980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ade6d762bf74ac0a3550a3943ebd322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:54.786945 17170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ade6d762bf74ac0a3550a3943ebd322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:54.794314 17412 tablet_bootstrap.cc:492] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: Bootstrap starting.
I20260808 11:43:54.796733 17412 tablet_bootstrap.cc:654] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:54.800595 17412 tablet_bootstrap.cc:492] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: No bootstrap required, opened a new log
I20260808 11:43:54.799937 17415 tablet_bootstrap.cc:492] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: Bootstrap starting.
I20260808 11:43:54.800808 17412 ts_tablet_manager.cc:1397] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 11:43:54.800948 17413 tablet_bootstrap.cc:492] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: Bootstrap starting.
I20260808 11:43:54.802156 17412 raft_consensus.cc:348] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.802563 17412 raft_consensus.cc:374] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:54.802798 17412 raft_consensus.cc:729] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Initialized, Role: FOLLOWER
I20260808 11:43:54.803205 17412 consensus_queue.cc:260] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.804141 17413 tablet_bootstrap.cc:654] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:54.804143 17415 tablet_bootstrap.cc:654] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:54.804150 17412 ts_tablet_manager.cc:1428] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 11:43:54.807983 17413 tablet_bootstrap.cc:492] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: No bootstrap required, opened a new log
I20260808 11:43:54.808168 17413 ts_tablet_manager.cc:1397] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 11:43:54.808307 17415 tablet_bootstrap.cc:492] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: No bootstrap required, opened a new log
I20260808 11:43:54.808531 17415 ts_tablet_manager.cc:1397] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260808 11:43:54.809405 17413 raft_consensus.cc:348] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.809463 17415 raft_consensus.cc:348] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.809697 17413 raft_consensus.cc:374] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:54.809790 17415 raft_consensus.cc:374] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:54.809862 17413 raft_consensus.cc:729] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79976a2ca0843d6bff7ac684dcaea1f, State: Initialized, Role: FOLLOWER
I20260808 11:43:54.809993 17415 raft_consensus.cc:729] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0378a2245e94da7b0610b5ba8ec63f3, State: Initialized, Role: FOLLOWER
I20260808 11:43:54.810174 17413 consensus_queue.cc:260] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.810304 17415 consensus_queue.cc:260] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.811575 17413 ts_tablet_manager.cc:1428] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 11:43:54.811743 17415 ts_tablet_manager.cc:1428] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260808 11:43:54.914870 17390 raft_consensus.cc:482] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:54.915213 17390 raft_consensus.cc:504] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.916277 17390 leader_election.cc:288] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:54.916931 16814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ade6d762bf74ac0a3550a3943ebd322" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" is_pre_election: true
I20260808 11:43:54.917008 17018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ade6d762bf74ac0a3550a3943ebd322" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" is_pre_election: true
I20260808 11:43:54.917354 16814 raft_consensus.cc:2454] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 0.
I20260808 11:43:54.917434 17018 raft_consensus.cc:2454] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 0.
I20260808 11:43:54.917972 17107 leader_election.cc:302] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a79976a2ca0843d6bff7ac684dcaea1f, d50bc29c8b00446a82f5e5082e8d6e37; no voters:
I20260808 11:43:54.918335 17390 raft_consensus.cc:2781] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:54.918548 17390 raft_consensus.cc:482] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:54.918728 17390 raft_consensus.cc:3037] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:54.921227 17390 raft_consensus.cc:504] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.921800 17390 leader_election.cc:288] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 election: Requested vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:54.922261 16814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ade6d762bf74ac0a3550a3943ebd322" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f"
I20260808 11:43:54.922389 17018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ade6d762bf74ac0a3550a3943ebd322" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378a2245e94da7b0610b5ba8ec63f3"
I20260808 11:43:54.922602 16814 raft_consensus.cc:3037] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:54.922664 17018 raft_consensus.cc:3037] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:54.925240 16814 raft_consensus.cc:2454] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 1.
I20260808 11:43:54.925297 17018 raft_consensus.cc:2454] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 1.
I20260808 11:43:54.925748 17107 leader_election.cc:302] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a79976a2ca0843d6bff7ac684dcaea1f, d50bc29c8b00446a82f5e5082e8d6e37; no voters:
I20260808 11:43:54.926091 17390 raft_consensus.cc:2781] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:54.926304 17390 raft_consensus.cc:686] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 LEADER]: Becoming Leader. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Running, Role: LEADER
I20260808 11:43:54.926633 17390 consensus_queue.cc:237] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:54.929957 16580 catalog_manager.cc:5495] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 reported cstate change: term changed from 0 to 1, leader changed from <none> to d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } health_report { overall_health: HEALTHY } } }
I20260808 11:43:54.968145 17425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:54.961603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.13.158.67:45423)
0808 11:43:54.962659 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:54.962663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:54.962691 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:43:54.962802 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:54.962806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:54.962890 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:54.962970 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:54.963424 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:54.964458 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:54.965917 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:54.966534 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:54.966645 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:54.967246 (+ 601us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:54.967361 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:54.967431 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:54.967561 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 11:43:54.967620 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":928,"thread_start_us":780,"threads_started":1}
I20260808 11:43:54.978534 17018 raft_consensus.cc:1261] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Refusing update from remote peer d50bc29c8b00446a82f5e5082e8d6e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:54.978624 16814 raft_consensus.cc:1261] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Refusing update from remote peer d50bc29c8b00446a82f5e5082e8d6e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:54.979244 17390 consensus_queue.cc:1035] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:54.979672 17298 consensus_queue.cc:1035] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:55.033630 16580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:55.033924 16580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:55.037102 16580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: Sending DeleteTablet for 3 replicas of tablet 2ade6d762bf74ac0a3550a3943ebd322
I20260808 11:43:55.037940 16783 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ade6d762bf74ac0a3550a3943ebd322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:55 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:52314
I20260808 11:43:55.038120 16980 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ade6d762bf74ac0a3550a3943ebd322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:55 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:35048
I20260808 11:43:55.038195 17170 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ade6d762bf74ac0a3550a3943ebd322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:55 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53162
I20260808 11:43:55.038331 17396 tablet_replica.cc:330] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: stopping tablet replica
I20260808 11:43:55.038493 17398 tablet_replica.cc:330] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: stopping tablet replica
I20260808 11:43:55.038496 17397 tablet_replica.cc:330] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: stopping tablet replica
I20260808 11:43:55.038659 17396 raft_consensus.cc:2229] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:55.038808 17398 raft_consensus.cc:2229] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:43:55.038808 17397 raft_consensus.cc:2229] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:55.038944 17396 raft_consensus.cc:2258] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:55.039055 17397 raft_consensus.cc:2258] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:55.039211 17398 raft_consensus.cc:2258] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:55.040043 17396 ts_tablet_manager.cc:1905] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:55.040282 17397 ts_tablet_manager.cc:1905] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:55.040282 17398 ts_tablet_manager.cc:1905] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:55.040416 16580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:43:55.041049 16580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:43:55.048676 17397 ts_tablet_manager.cc:1918] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:55.048664 17396 ts_tablet_manager.cc:1918] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:55.048900 17397 log.cc:1196] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/2ade6d762bf74ac0a3550a3943ebd322
I20260808 11:43:55.049044 17396 log.cc:1196] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/2ade6d762bf74ac0a3550a3943ebd322
I20260808 11:43:55.049551 17397 ts_tablet_manager.cc:1939] T 2ade6d762bf74ac0a3550a3943ebd322 P f0378a2245e94da7b0610b5ba8ec63f3: Deleting consensus metadata
I20260808 11:43:55.049713 17396 ts_tablet_manager.cc:1939] T 2ade6d762bf74ac0a3550a3943ebd322 P a79976a2ca0843d6bff7ac684dcaea1f: Deleting consensus metadata
I20260808 11:43:55.050635 17170 tablet_service.cc:1467] Processing CreateTablet for tablet cec158562d8142bbb4ca9a1051fa2f3b (DEFAULT_TABLE table=test-table [id=09303365f0394fdbab2851aef1f2e045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:55.050855 16556 catalog_manager.cc:4841] TS f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457): tablet 2ade6d762bf74ac0a3550a3943ebd322 (table test-table [id=6ad3c56c379c41218ddecc147d2b52be]) successfully deleted
I20260808 11:43:55.051175 17170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec158562d8142bbb4ca9a1051fa2f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:55.051122 16980 tablet_service.cc:1467] Processing CreateTablet for tablet cec158562d8142bbb4ca9a1051fa2f3b (DEFAULT_TABLE table=test-table [id=09303365f0394fdbab2851aef1f2e045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:55.051592 16980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec158562d8142bbb4ca9a1051fa2f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:55.053057 17398 ts_tablet_manager.cc:1918] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:43:55.053823 16558 catalog_manager.cc:4841] TS a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345): tablet 2ade6d762bf74ac0a3550a3943ebd322 (table test-table [id=6ad3c56c379c41218ddecc147d2b52be]) successfully deleted
I20260808 11:43:55.055092 17398 log.cc:1196] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/2ade6d762bf74ac0a3550a3943ebd322
I20260808 11:43:55.055321 16783 tablet_service.cc:1467] Processing CreateTablet for tablet cec158562d8142bbb4ca9a1051fa2f3b (DEFAULT_TABLE table=test-table [id=09303365f0394fdbab2851aef1f2e045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:43:55.055812 16783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec158562d8142bbb4ca9a1051fa2f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:55.056114 17398 ts_tablet_manager.cc:1939] T 2ade6d762bf74ac0a3550a3943ebd322 P d50bc29c8b00446a82f5e5082e8d6e37: Deleting consensus metadata
I20260808 11:43:55.058092 16558 catalog_manager.cc:4841] TS d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67:45423): tablet 2ade6d762bf74ac0a3550a3943ebd322 (table test-table [id=6ad3c56c379c41218ddecc147d2b52be]) successfully deleted
I20260808 11:43:55.059329 17412 tablet_bootstrap.cc:492] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: Bootstrap starting.
I20260808 11:43:55.059330 17415 tablet_bootstrap.cc:492] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: Bootstrap starting.
I20260808 11:43:55.061160 17415 tablet_bootstrap.cc:654] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:55.061941 17412 tablet_bootstrap.cc:654] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:55.064643 17413 tablet_bootstrap.cc:492] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: Bootstrap starting.
I20260808 11:43:55.065218 17415 tablet_bootstrap.cc:492] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: No bootstrap required, opened a new log
I20260808 11:43:55.065502 17415 ts_tablet_manager.cc:1397] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 11:43:55.066267 17412 tablet_bootstrap.cc:492] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: No bootstrap required, opened a new log
I20260808 11:43:55.066525 17412 ts_tablet_manager.cc:1397] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260808 11:43:55.066634 17413 tablet_bootstrap.cc:654] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:55.066663 17415 raft_consensus.cc:348] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.067225 17415 raft_consensus.cc:374] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:55.067359 17415 raft_consensus.cc:729] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0378a2245e94da7b0610b5ba8ec63f3, State: Initialized, Role: FOLLOWER
I20260808 11:43:55.067713 17415 consensus_queue.cc:260] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.067786 17412 raft_consensus.cc:348] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.068112 17412 raft_consensus.cc:374] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:55.068243 17412 raft_consensus.cc:729] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Initialized, Role: FOLLOWER
I20260808 11:43:55.068594 17415 ts_tablet_manager.cc:1428] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:55.068579 17412 consensus_queue.cc:260] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.069193 17412 ts_tablet_manager.cc:1428] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260808 11:43:55.071110 17413 tablet_bootstrap.cc:492] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: No bootstrap required, opened a new log
I20260808 11:43:55.071362 17413 ts_tablet_manager.cc:1397] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260808 11:43:55.073078 17413 raft_consensus.cc:348] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.073446 17413 raft_consensus.cc:374] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:55.073588 17413 raft_consensus.cc:729] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79976a2ca0843d6bff7ac684dcaea1f, State: Initialized, Role: FOLLOWER
I20260808 11:43:55.073922 17413 consensus_queue.cc:260] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.074570 17413 ts_tablet_manager.cc:1428] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:55.261477 17298 raft_consensus.cc:482] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:43:55.261909 17298 raft_consensus.cc:504] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.262666 17298 leader_election.cc:288] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:55.263293 16815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec158562d8142bbb4ca9a1051fa2f3b" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" is_pre_election: true
I20260808 11:43:55.263485 17018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec158562d8142bbb4ca9a1051fa2f3b" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" is_pre_election: true
I20260808 11:43:55.263631 16815 raft_consensus.cc:2454] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 0.
I20260808 11:43:55.263850 17018 raft_consensus.cc:2454] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 0.
I20260808 11:43:55.264248 17107 leader_election.cc:302] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a79976a2ca0843d6bff7ac684dcaea1f, d50bc29c8b00446a82f5e5082e8d6e37; no voters:
I20260808 11:43:55.264628 17298 raft_consensus.cc:2781] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:43:55.264854 17298 raft_consensus.cc:482] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:43:55.265031 17298 raft_consensus.cc:3037] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:55.269292 17298 raft_consensus.cc:504] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.270469 17298 leader_election.cc:288] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 election: Requested vote from peers a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345), f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457)
I20260808 11:43:55.270812 17018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec158562d8142bbb4ca9a1051fa2f3b" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0378a2245e94da7b0610b5ba8ec63f3"
I20260808 11:43:55.270773 16815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec158562d8142bbb4ca9a1051fa2f3b" candidate_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a79976a2ca0843d6bff7ac684dcaea1f"
I20260808 11:43:55.271080 17018 raft_consensus.cc:3037] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:55.271178 16815 raft_consensus.cc:3037] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:55.275210 16815 raft_consensus.cc:2454] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 1.
I20260808 11:43:55.275264 17018 raft_consensus.cc:2454] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50bc29c8b00446a82f5e5082e8d6e37 in term 1.
I20260808 11:43:55.275722 17105 leader_election.cc:302] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d50bc29c8b00446a82f5e5082e8d6e37, f0378a2245e94da7b0610b5ba8ec63f3; no voters:
I20260808 11:43:55.276113 17298 raft_consensus.cc:2781] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:55.276422 17298 raft_consensus.cc:686] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 LEADER]: Becoming Leader. State: Replica: d50bc29c8b00446a82f5e5082e8d6e37, State: Running, Role: LEADER
I20260808 11:43:55.276757 17298 consensus_queue.cc:237] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } }
I20260808 11:43:55.279589 16576 catalog_manager.cc:5495] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 reported cstate change: term changed from 0 to 1, leader changed from <none> to d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d50bc29c8b00446a82f5e5082e8d6e37" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45423 } health_report { overall_health: HEALTHY } } }
I20260808 11:43:55.311355 17018 raft_consensus.cc:1261] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Refusing update from remote peer d50bc29c8b00446a82f5e5082e8d6e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:55.311873 16815 raft_consensus.cc:1261] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Refusing update from remote peer d50bc29c8b00446a82f5e5082e8d6e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:55.312052 17374 consensus_queue.cc:1035] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0378a2245e94da7b0610b5ba8ec63f3" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:55.312578 17390 consensus_queue.cc:1035] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a79976a2ca0843d6bff7ac684dcaea1f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:43:55.333581 17372 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 11:43:56.920769 16580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:56.921092 16580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:43:56.926039 16580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eed45be70b842d785d268b3c6ece825: Sending DeleteTablet for 3 replicas of tablet cec158562d8142bbb4ca9a1051fa2f3b
I20260808 11:43:56.926992 16783 tablet_service.cc:1514] Processing DeleteTablet for tablet cec158562d8142bbb4ca9a1051fa2f3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:56 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:52314
I20260808 11:43:56.927173 16980 tablet_service.cc:1514] Processing DeleteTablet for tablet cec158562d8142bbb4ca9a1051fa2f3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:56 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:35048
I20260808 11:43:56.927698 17170 tablet_service.cc:1514] Processing DeleteTablet for tablet cec158562d8142bbb4ca9a1051fa2f3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:43:56 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53162
I20260808 11:43:56.927842 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 16688
I20260808 11:43:56.928102 17469 tablet_replica.cc:330] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: stopping tablet replica
I20260808 11:43:56.928483 17469 raft_consensus.cc:2229] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:56.928881 17469 raft_consensus.cc:2258] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:56.929809 17471 tablet_replica.cc:330] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: stopping tablet replica
I20260808 11:43:56.930140 17471 raft_consensus.cc:2229] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:43:56.930791 17471 raft_consensus.cc:2258] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:56.931999 17469 ts_tablet_manager.cc:1905] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:56.935616 17471 ts_tablet_manager.cc:1905] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:56.939622 17470 tablet_replica.cc:330] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: stopping tablet replica
I20260808 11:43:56.940129 17470 raft_consensus.cc:2229] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:43:56.940649 17470 raft_consensus.cc:2258] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:43:56.942323 17470 ts_tablet_manager.cc:1905] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:43:56.945009 17471 ts_tablet_manager.cc:1918] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:43:56.945225 17471 log.cc:1196] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/cec158562d8142bbb4ca9a1051fa2f3b
I20260808 11:43:56.945757 17471 ts_tablet_manager.cc:1939] T cec158562d8142bbb4ca9a1051fa2f3b P d50bc29c8b00446a82f5e5082e8d6e37: Deleting consensus metadata
I20260808 11:43:56.947000 16558 catalog_manager.cc:4841] TS d50bc29c8b00446a82f5e5082e8d6e37 (127.13.158.67:45423): tablet cec158562d8142bbb4ca9a1051fa2f3b (table test-table [id=09303365f0394fdbab2851aef1f2e045]) successfully deleted
I20260808 11:43:56.949465 17469 ts_tablet_manager.cc:1918] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:43:56.949759 17469 log.cc:1196] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/cec158562d8142bbb4ca9a1051fa2f3b
I20260808 11:43:56.950132 17470 ts_tablet_manager.cc:1918] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:43:56.950413 17470 log.cc:1196] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/cec158562d8142bbb4ca9a1051fa2f3b
I20260808 11:43:56.950444 17469 ts_tablet_manager.cc:1939] T cec158562d8142bbb4ca9a1051fa2f3b P a79976a2ca0843d6bff7ac684dcaea1f: Deleting consensus metadata
I20260808 11:43:56.951144 17470 ts_tablet_manager.cc:1939] T cec158562d8142bbb4ca9a1051fa2f3b P f0378a2245e94da7b0610b5ba8ec63f3: Deleting consensus metadata
I20260808 11:43:56.951573 16558 catalog_manager.cc:4841] TS a79976a2ca0843d6bff7ac684dcaea1f (127.13.158.65:39345): tablet cec158562d8142bbb4ca9a1051fa2f3b (table test-table [id=09303365f0394fdbab2851aef1f2e045]) successfully deleted
I20260808 11:43:56.952458 16556 catalog_manager.cc:4841] TS f0378a2245e94da7b0610b5ba8ec63f3 (127.13.158.66:34457): tablet cec158562d8142bbb4ca9a1051fa2f3b (table test-table [id=09303365f0394fdbab2851aef1f2e045]) successfully deleted
Aug 08 11:43:56 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189434, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:43:56.971566 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:56.929664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.13.158.65:39345)
0808 11:43:56.940758 (+ 11094us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:56.940769 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:56.945817 (+ 5048us) server_negotiation.cc:407] Connection header received
0808 11:43:56.946448 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:56.946455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:56.946605 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:56.946768 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:56.949489 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:56.951292 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:56.953268 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:56.953823 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:56.953926 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:56.963972 (+ 10046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:56.963989 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:56.964007 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:56.964057 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:56.968032 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:56.968769 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:56.968779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:56.968783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:56.969011 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:56.969485 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:56.969491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:56.969494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:56.969927 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:56.970316 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:56.970686 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 11:43:56.971020 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10891,"thread_start_us":10732,"threads_started":1}
W20260808 11:43:56.972584 16877 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52322
W20260808 11:43:56.973315 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 16688: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:56.973686 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 16688
I20260808 11:43:57.001637 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 16889
Aug 08 11:43:57 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189434, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:43:57.017135 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:57.002600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.13.158.66:34457)
0808 11:43:57.003294 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:57.003298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:57.003314 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:43:57.003368 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:57.003372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:57.003427 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:57.003541 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:57.005024 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:57.005783 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:57.007068 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:57.007482 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:57.007582 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:57.011820 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:57.011836 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:57.011853 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:57.011885 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:57.014586 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:57.015356 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:57.015363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:57.015366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:57.015487 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:57.015815 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:57.015818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:57.015820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:57.016145 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:57.016378 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:57.016586 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 11:43:57.016760 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":420,"threads_started":1}
W20260808 11:43:57.017897 17072 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35068
W20260808 11:43:57.018646 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 16889: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:57.018826 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 16889
I20260808 11:43:57.051620 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 17085
Aug 08 11:43:57 dist-test-slave-39l8 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189434, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:43:57.066941 17477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:57.052543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53194 (local address 127.13.158.67:45423)
0808 11:43:57.053075 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:57.053080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:57.053096 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:43:57.053152 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:57.053156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:57.053228 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:57.053321 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:57.054580 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:57.055356 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:57.056680 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:57.057109 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:57.057270 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:57.061748 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:57.061762 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:57.061780 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:57.061812 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:57.064563 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:57.065184 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:57.065192 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:57.065194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:57.065312 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:57.065653 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:57.065655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:57.065657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:57.065991 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:57.066198 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:57.066448 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 11:43:57.066619 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":287,"threads_started":1}
W20260808 11:43:57.067590 17252 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53194
W20260808 11:43:57.068223 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 17085: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:57.068367 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 17085
I20260808 11:43:57.104689 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 16532
I20260808 11:43:57.105978 16641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:57.221668 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 16532
2026-08-08T11:43:57Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8516 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:57 dist-test-slave-39l8 krb5kdc[17481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:57 dist-test-slave-39l8 krb5kdc[17481](info): set up 2 sockets
Aug 08 11:43:57 dist-test-slave-39l8 krb5kdc[17481](info): commencing operation
krb5kdc: starting...
W20260808 11:43:58.502827 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:43:58 dist-test-slave-39l8 krb5kdc[17481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:43:58.710388 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:32823
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:59.034602 17496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:59.035032 17496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:59.035210 17496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:59.042192 17496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:59.042339 17496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:59.042455 17496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:59.042564 17496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:59.042655 17496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:59.052630 17496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:32823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:59.053961 17496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:59.055330 17496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:59.067982 17503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:59.068043 17505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:59.068045 17509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:59.068486 17496 server_base.cc:1034] running on GCE node
I20260808 11:43:59.069320 17496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:59.070693 17496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:59.072070 17496 hybrid_clock.cc:648] HybridClock initialized: now 1786189439072025 us; error 28 us; skew 500 ppm
I20260808 11:43:59.072463 17496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:59 dist-test-slave-39l8 krb5kdc[17481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189439, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:59.075559 17496 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:43:59.077004 17496 webserver.cc:460] Webserver started at http://127.13.158.126:42841/ using document root <none> and password file <none>
I20260808 11:43:59.077733 17496 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:59.077885 17496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:59.078212 17496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:59.081187 17496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "da3e4d1180fd4720ae631f29ece3aced"
format_stamp: "Formatted at 2026-08-08 11:43:59 on dist-test-slave-39l8"
server_key: "e4c17dacda284d54ef9662fdeef1766a"
server_key_iv: "58f5d58d5a8ac1d04dccd25b3efbb7da"
server_key_version: "encryptionkey@0"
I20260808 11:43:59.081928 17496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "da3e4d1180fd4720ae631f29ece3aced"
format_stamp: "Formatted at 2026-08-08 11:43:59 on dist-test-slave-39l8"
server_key: "e4c17dacda284d54ef9662fdeef1766a"
server_key_iv: "58f5d58d5a8ac1d04dccd25b3efbb7da"
server_key_version: "encryptionkey@0"
I20260808 11:43:59.087157 17496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 11:43:59.090513 17517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:59.091799 17496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:59.091991 17496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "da3e4d1180fd4720ae631f29ece3aced"
format_stamp: "Formatted at 2026-08-08 11:43:59 on dist-test-slave-39l8"
server_key: "e4c17dacda284d54ef9662fdeef1766a"
server_key_iv: "58f5d58d5a8ac1d04dccd25b3efbb7da"
server_key_version: "encryptionkey@0"
I20260808 11:43:59.092208 17496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:59.120214 17496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:59.124310 17496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:59.124606 17496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:59.143326 17496 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:32823
I20260808 11:43:59.143344 17587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:32823 every 8 connection(s)
I20260808 11:43:59.144737 17496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:59.148291 17588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:59.150695 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 17496
I20260808 11:43:59.150980 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:43:59.151765 13945 external_mini_cluster.cc:1428] Setting key ceeb5786f002677ec5bc48d7c4db5c40
I20260808 11:43:59.160490 17588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced: Bootstrap starting.
I20260808 11:43:59.163583 17588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:59.164569 17588 log.cc:824] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:43:59 dist-test-slave-39l8 krb5kdc[17481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:59.167811 17588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced: No bootstrap required, opened a new log
I20260808 11:43:59.175468 17595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:59.153903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.13.158.126:32823)
0808 11:43:59.155574 (+ 1671us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:59.155585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:59.155625 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:43:59.157254 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:59.157275 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:59.157634 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:59.158024 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:59.159569 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:59.160943 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:59.162337 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:59.163053 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:59.163280 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:59.168046 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:59.168053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:59.168066 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:59.168100 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:59.171298 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:59.172150 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:59.172156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:59.172162 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:59.172330 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:59.172746 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:59.172749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:59.172751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:59.173281 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:59.173507 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:59.173969 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 11:43:59.174352 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1478,"thread_start_us":1255,"threads_started":1}
I20260808 11:43:59.176882 17588 raft_consensus.cc:348] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3e4d1180fd4720ae631f29ece3aced" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32823 } }
I20260808 11:43:59.177235 17588 raft_consensus.cc:374] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:59.177369 17588 raft_consensus.cc:729] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3e4d1180fd4720ae631f29ece3aced, State: Initialized, Role: FOLLOWER
I20260808 11:43:59.177981 17588 consensus_queue.cc:260] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3e4d1180fd4720ae631f29ece3aced" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32823 } }
I20260808 11:43:59.178221 17588 raft_consensus.cc:388] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:59.178372 17588 raft_consensus.cc:482] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:59.178560 17588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:59.180653 17588 raft_consensus.cc:504] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3e4d1180fd4720ae631f29ece3aced" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32823 } }
I20260808 11:43:59.181143 17588 leader_election.cc:302] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da3e4d1180fd4720ae631f29ece3aced; no voters:
I20260808 11:43:59.181751 17588 leader_election.cc:288] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:59.181839 17597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:59.182629 17597 raft_consensus.cc:686] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [term 1 LEADER]: Becoming Leader. State: Replica: da3e4d1180fd4720ae631f29ece3aced, State: Running, Role: LEADER
I20260808 11:43:59.183171 17597 consensus_queue.cc:237] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3e4d1180fd4720ae631f29ece3aced" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32823 } }
I20260808 11:43:59.183873 17588 sys_catalog.cc:564] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:59.185822 17600 sys_catalog.cc:455] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [sys.catalog]: SysCatalogTable state changed. Reason: New leader da3e4d1180fd4720ae631f29ece3aced. Latest consensus state: current_term: 1 leader_uuid: "da3e4d1180fd4720ae631f29ece3aced" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3e4d1180fd4720ae631f29ece3aced" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32823 } } }
I20260808 11:43:59.186304 17599 sys_catalog.cc:455] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da3e4d1180fd4720ae631f29ece3aced" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3e4d1180fd4720ae631f29ece3aced" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32823 } } }
I20260808 11:43:59.186578 17599 sys_catalog.cc:458] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:59.186872 17600 sys_catalog.cc:458] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:59.188648 17607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:59.193485 17607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:59.202920 17607 catalog_manager.cc:1347] Generated new cluster ID: fbba3b827788471b8d91284319a1b44f
I20260808 11:43:59.203045 17607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:59.223765 17607 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:59.225165 17607 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:59.235849 17607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da3e4d1180fd4720ae631f29ece3aced: Generated new TSK 0
I20260808 11:43:59.236795 17607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:43:59.292768 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:32823
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:59.621240 17635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:59.621521 17635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:59.621613 17635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:59.629072 17635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:59.629213 17635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:59.629339 17635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:43:59.639258 17635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:59.640497 17635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:59.641767 17635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:59.654646 17641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:59.654800 17646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:59.654687 17644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:59.655915 17635 server_base.cc:1034] running on GCE node
I20260808 11:43:59.656577 17635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:59.657943 17635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:59.659255 17635 hybrid_clock.cc:648] HybridClock initialized: now 1786189439659129 us; error 97 us; skew 500 ppm
I20260808 11:43:59.659682 17635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:59 dist-test-slave-39l8 krb5kdc[17481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189439, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:59.663058 17635 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:43:59.664506 17635 webserver.cc:460] Webserver started at http://127.13.158.65:42521/ using document root <none> and password file <none>
I20260808 11:43:59.665237 17635 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:59.665390 17635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:59.665723 17635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:59.668923 17635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "9c0121aba5ae419984f69465d052f312"
format_stamp: "Formatted at 2026-08-08 11:43:59 on dist-test-slave-39l8"
server_key: "12b6b1f203402b1c282a3abb907cab35"
server_key_iv: "998f015d4a167bc7937eea679b84601f"
server_key_version: "encryptionkey@0"
I20260808 11:43:59.669741 17635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "9c0121aba5ae419984f69465d052f312"
format_stamp: "Formatted at 2026-08-08 11:43:59 on dist-test-slave-39l8"
server_key: "12b6b1f203402b1c282a3abb907cab35"
server_key_iv: "998f015d4a167bc7937eea679b84601f"
server_key_version: "encryptionkey@0"
I20260808 11:43:59.675496 17635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:43:59.679114 17655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:59.680434 17635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:43:59.680622 17635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "9c0121aba5ae419984f69465d052f312"
format_stamp: "Formatted at 2026-08-08 11:43:59 on dist-test-slave-39l8"
server_key: "12b6b1f203402b1c282a3abb907cab35"
server_key_iv: "998f015d4a167bc7937eea679b84601f"
server_key_version: "encryptionkey@0"
I20260808 11:43:59.680836 17635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:59.700922 17635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:59.704898 17635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:59.705217 17635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:59.706209 17635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:43:59.707723 17635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:43:59.707836 17635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:59.707994 17635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:43:59.708070 17635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:59.741840 17635 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:43545
I20260808 11:43:59.741891 17813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:43545 every 8 connection(s)
I20260808 11:43:59.743299 17635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:43:59.753686 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 17635
I20260808 11:43:59.753917 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:43:59.754783 13945 external_mini_cluster.cc:1428] Setting key 389c9bd8296a013602001091ba56811f
Aug 08 11:43:59 dist-test-slave-39l8 krb5kdc[17481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189439, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:43:59.763720 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:59.745900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:48373 (local address 127.13.158.126:32823)
0808 11:43:59.746348 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:59.746353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:59.747077 (+ 724us) server_negotiation.cc:407] Connection header received
0808 11:43:59.748817 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:59.748823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:59.748906 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:59.749017 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:59.750471 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:59.751296 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:59.752855 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:59.753350 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:59.753493 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:59.757679 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:59.757692 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:59.757695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:59.757725 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:59.760326 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:59.761303 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:59.761307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:59.761309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:59.761424 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:59.761901 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:59.761904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:59.761906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:59.762147 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:59.762258 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:59.763158 (+ 900us) server_negotiation.cc:299] Negotiation successful
0808 11:43:59.763329 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260808 11:43:59.764735 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:59.746202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:32823 (local address 127.13.158.65:48373)
0808 11:43:59.746923 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:59.746968 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:59.748541 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:59.749275 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:59.749282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:59.749769 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:59.750291 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:59.750307 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:59.751504 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:59.751508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:59.752700 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:59.752722 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:59.753650 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:59.753655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:59.753917 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:59.754698 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:59.754778 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:59.757453 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:59.760474 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:59.760485 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:59.760502 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:59.761182 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:59.761534 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:59.761542 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:59.761548 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:59.761793 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:59.762272 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:59.762282 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:59.762751 (+ 469us) client_negotiation.cc:769] Sending connection context
0808 11:43:59.763060 (+ 309us) client_negotiation.cc:240] Negotiation successful
0808 11:43:59.763598 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":257,"threads_started":1}
I20260808 11:43:59.766338 17814 heartbeater.cc:347] Connected to a master server at 127.13.158.126:32823
I20260808 11:43:59.766635 17814 heartbeater.cc:464] Registering TS with master...
I20260808 11:43:59.767397 17814 heartbeater.cc:511] Master 127.13.158.126:32823 requested a full tablet report, sending...
I20260808 11:43:59.769625 17540 ts_manager.cc:194] Registered new tserver with Master: 9c0121aba5ae419984f69465d052f312 (127.13.158.65:43545)
I20260808 11:43:59.771932 17540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:48373
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:43:59.801939 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:32823
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:00.132727 17821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:00.133023 17821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:00.133117 17821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:00.140475 17821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:00.140584 17821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:00.140698 17821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:00.150499 17821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:00.151760 17821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:00.153045 17821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:00.166363 17826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:00.166317 17827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:00.166299 17830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:00.167279 17821 server_base.cc:1034] running on GCE node
I20260808 11:44:00.167925 17821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:00.168866 17821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:00.170113 17821 hybrid_clock.cc:648] HybridClock initialized: now 1786189440170053 us; error 68 us; skew 500 ppm
I20260808 11:44:00.170444 17821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:00 dist-test-slave-39l8 krb5kdc[17481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189440, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:00.173661 17821 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:44:00.175123 17821 webserver.cc:460] Webserver started at http://127.13.158.66:39259/ using document root <none> and password file <none>
I20260808 11:44:00.175849 17821 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:00.175951 17821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:00.176199 17821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:00.179214 17821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "61693731c6e246e08fa4e599c756b87e"
format_stamp: "Formatted at 2026-08-08 11:44:00 on dist-test-slave-39l8"
server_key: "7638cdbac5ac79a33e9685dd1e0b8db9"
server_key_iv: "3be826d36eff45b7abee5b69d893da38"
server_key_version: "encryptionkey@0"
I20260808 11:44:00.179934 17821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "61693731c6e246e08fa4e599c756b87e"
format_stamp: "Formatted at 2026-08-08 11:44:00 on dist-test-slave-39l8"
server_key: "7638cdbac5ac79a33e9685dd1e0b8db9"
server_key_iv: "3be826d36eff45b7abee5b69d893da38"
server_key_version: "encryptionkey@0"
I20260808 11:44:00.185182 17821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 11:44:00.188685 17838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:00.190070 17821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:44:00.190217 17821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "61693731c6e246e08fa4e599c756b87e"
format_stamp: "Formatted at 2026-08-08 11:44:00 on dist-test-slave-39l8"
server_key: "7638cdbac5ac79a33e9685dd1e0b8db9"
server_key_iv: "3be826d36eff45b7abee5b69d893da38"
server_key_version: "encryptionkey@0"
I20260808 11:44:00.190443 17821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:00.207908 17821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:00.211849 17821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:00.212107 17821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:00.213007 17821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:00.214272 17821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:00.214342 17821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:00.214443 17821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:00.214486 17821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:00.250936 17821 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:42159
I20260808 11:44:00.250958 17995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:42159 every 8 connection(s)
I20260808 11:44:00.252298 17821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:00.261775 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 17821
I20260808 11:44:00.261987 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:00.262456 13945 external_mini_cluster.cc:1428] Setting key 5c12e790ef86538914bcaff73421a793
Aug 08 11:44:00 dist-test-slave-39l8 krb5kdc[17481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189440, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:00.273736 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:00.254910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:46367 (local address 127.13.158.126:32823)
0808 11:44:00.255095 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:00.255100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:00.256220 (+ 1120us) server_negotiation.cc:407] Connection header received
0808 11:44:00.257996 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:00.258000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:00.258069 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:00.258160 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:00.259654 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:00.260355 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:00.262148 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:00.262619 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:00.262803 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:00.266821 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:00.266830 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:00.266838 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:00.266886 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:00.270474 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:00.271249 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:00.271256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:00.271262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:00.271418 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:00.271850 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:00.271857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:00.271863 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:00.272262 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:00.272409 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:00.273303 (+ 894us) server_negotiation.cc:299] Negotiation successful
0808 11:44:00.273480 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:44:00.275146 18001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:00.255323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:32823 (local address 127.13.158.66:46367)
0808 11:44:00.256045 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:00.256093 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:00.257740 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:00.258371 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:00.258378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:00.258951 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:00.259507 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:00.259523 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:00.260485 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:00.260488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:00.262007 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:00.262024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:00.262918 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:00.262926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:00.263201 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:00.264017 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:00.264046 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:00.266552 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:00.270621 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:00.270628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:00.270640 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:00.271124 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:00.271530 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:00.271533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:00.271535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:00.271726 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:00.272416 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:00.272427 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:00.272888 (+ 461us) client_negotiation.cc:769] Sending connection context
0808 11:44:00.273229 (+ 341us) client_negotiation.cc:240] Negotiation successful
0808 11:44:00.273827 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":264,"threads_started":1}
I20260808 11:44:00.276669 17997 heartbeater.cc:347] Connected to a master server at 127.13.158.126:32823
I20260808 11:44:00.276983 17997 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:00.277705 17997 heartbeater.cc:511] Master 127.13.158.126:32823 requested a full tablet report, sending...
I20260808 11:44:00.279234 17540 ts_manager.cc:194] Registered new tserver with Master: 61693731c6e246e08fa4e599c756b87e (127.13.158.66:42159)
I20260808 11:44:00.280076 17540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:46367
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:44:00.311455 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:32823
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:00.637686 18005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:00.638028 18005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:00.638211 18005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:00.645431 18005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:00.645592 18005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:00.645787 18005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:00.655722 18005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:00.657123 18005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:00.658478 18005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:00.672940 18011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:00.672896 18012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:00.672969 18015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:00.674010 18005 server_base.cc:1034] running on GCE node
I20260808 11:44:00.674582 18005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:00.675498 18005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:00.676733 18005 hybrid_clock.cc:648] HybridClock initialized: now 1786189440676681 us; error 57 us; skew 500 ppm
I20260808 11:44:00.677047 18005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:00 dist-test-slave-39l8 krb5kdc[17481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189440, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:00.680132 18005 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:44:00.681500 18005 webserver.cc:460] Webserver started at http://127.13.158.67:41501/ using document root <none> and password file <none>
I20260808 11:44:00.682222 18005 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:00.682365 18005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:00.682760 18005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:00.685870 18005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "510810cedc844f04beb46942fca8d4ef"
format_stamp: "Formatted at 2026-08-08 11:44:00 on dist-test-slave-39l8"
server_key: "56da42a5caa01433dfdabd729c4b0f2e"
server_key_iv: "bb4e1ea83c31512bf4373413ad2c86df"
server_key_version: "encryptionkey@0"
I20260808 11:44:00.686653 18005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "510810cedc844f04beb46942fca8d4ef"
format_stamp: "Formatted at 2026-08-08 11:44:00 on dist-test-slave-39l8"
server_key: "56da42a5caa01433dfdabd729c4b0f2e"
server_key_iv: "bb4e1ea83c31512bf4373413ad2c86df"
server_key_version: "encryptionkey@0"
I20260808 11:44:00.691933 18005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:44:00.695533 18030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:00.697003 18005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:44:00.697180 18005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "510810cedc844f04beb46942fca8d4ef"
format_stamp: "Formatted at 2026-08-08 11:44:00 on dist-test-slave-39l8"
server_key: "56da42a5caa01433dfdabd729c4b0f2e"
server_key_iv: "bb4e1ea83c31512bf4373413ad2c86df"
server_key_version: "encryptionkey@0"
I20260808 11:44:00.697377 18005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:00.724203 18005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:00.728526 18005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:00.728863 18005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:00.729916 18005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:00.731374 18005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:00.731484 18005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:00.731642 18005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:00.731725 18005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:00.766256 18005 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:41327
I20260808 11:44:00.766312 18205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:41327 every 8 connection(s)
I20260808 11:44:00.767617 18005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:00.772177 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 18005
I20260808 11:44:00.772395 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:00.772951 13945 external_mini_cluster.cc:1428] Setting key 7cf0688fe08a3e19f5f09758b6612504
I20260808 11:44:00.774863 17814 heartbeater.cc:503] Master 127.13.158.126:32823 was elected leader, sending a full tablet report...
Aug 08 11:44:00 dist-test-slave-39l8 krb5kdc[17481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189440, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:00.788535 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:00.770296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:36651 (local address 127.13.158.126:32823)
0808 11:44:00.770483 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:00.770487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:00.771658 (+ 1171us) server_negotiation.cc:407] Connection header received
0808 11:44:00.774094 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:00.774099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:00.774184 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:00.774300 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:00.776347 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:00.777177 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:00.778639 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:00.779109 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:00.779211 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:00.782342 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:00.782347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:00.782350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:00.782381 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:00.785401 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:00.786071 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:00.786075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:00.786077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:00.786200 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:00.786609 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:00.786612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:00.786614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:00.787002 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:00.787110 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:00.787955 (+ 845us) server_negotiation.cc:299] Negotiation successful
0808 11:44:00.788168 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:44:00.789487 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:00.770751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:32823 (local address 127.13.158.67:36651)
0808 11:44:00.771479 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:00.771528 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:00.773792 (+ 2264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:00.774530 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:00.774556 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:00.775361 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:00.776135 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:00.776179 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:00.777310 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:00.777314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:00.778515 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:00.778527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:00.779300 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:00.779305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:00.779491 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:00.780097 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:00.780119 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:00.782155 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:00.785539 (+ 3384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:00.785545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:00.785556 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:00.785956 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:00.786292 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:00.786295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:00.786297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:00.786505 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:00.787122 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:00.787130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:00.787524 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 11:44:00.787821 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 11:44:00.788281 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":274,"threads_started":1}
I20260808 11:44:00.790864 18206 heartbeater.cc:347] Connected to a master server at 127.13.158.126:32823
I20260808 11:44:00.791185 18206 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:00.791874 18206 heartbeater.cc:511] Master 127.13.158.126:32823 requested a full tablet report, sending...
I20260808 11:44:00.793319 17539 ts_manager.cc:194] Registered new tserver with Master: 510810cedc844f04beb46942fca8d4ef (127.13.158.67:41327)
I20260808 11:44:00.794222 17539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:36651
I20260808 11:44:00.800542 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 11:44:00.834646 17816 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:00.832208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.13.158.126:32823)
0808 11:44:00.832393 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:00.832397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:00.832849 (+ 452us) server_negotiation.cc:407] Connection header received
0808 11:44:00.833000 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:00.833004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:00.833068 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:00.833165 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:00.834394 (+ 1229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40474: BlockingRecv error: recv got EOF from 127.0.0.1:40474 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:44:00.837780 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 17635
W20260808 11:44:00.840664 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 17635: Not authorized: Client connection negotiation failed: client connection to 127.13.158.65:43545: server requires authentication, but client does not have Kerberos credentials available
I20260808 11:44:00.840839 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 17635
I20260808 11:44:00.863536 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 17821
W20260808 11:44:00.866518 18219 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:00.864409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.13.158.66:42159)
0808 11:44:00.864925 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:00.864950 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:00.864975 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:44:00.865064 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:00.865071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:00.865280 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:00.865462 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:00.866237 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39102: BlockingRecv error: recv got EOF from 127.0.0.1:39102 (error 108)
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":233,"threads_started":1}
W20260808 11:44:00.866873 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 17821: Not authorized: Client connection negotiation failed: client connection to 127.13.158.66:42159: server requires authentication, but client does not have Kerberos credentials available
I20260808 11:44:00.867053 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 17821
I20260808 11:44:00.890343 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18005
W20260808 11:44:00.893714 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18005: Not authorized: Client connection negotiation failed: client connection to 127.13.158.67:41327: server requires authentication, but client does not have Kerberos credentials available
I20260808 11:44:00.893872 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 18005
I20260808 11:44:00.914909 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 17496
I20260808 11:44:00.916289 17581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:01.017702 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 17496
2026-08-08T11:44:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3791 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:01 dist-test-slave-39l8 krb5kdc[18229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:01 dist-test-slave-39l8 krb5kdc[18229](info): set up 2 sockets
Aug 08 11:44:01 dist-test-slave-39l8 krb5kdc[18229](info): commencing operation
krb5kdc: starting...
W20260808 11:44:02.291669 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:02 dist-test-slave-39l8 krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:44:02.498405 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36035
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:36035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:02.826654 18248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:02.826951 18248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:02.827147 18248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:02.834342 18248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:02.834444 18248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:02.834491 18248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:02.834569 18248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:02.834643 18248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:02.844316 18248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:36035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:02.845571 18248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:02.846864 18248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:02.858328 18253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:02.858428 18257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:02.858675 18261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:02.859632 18248 server_base.cc:1034] running on GCE node
I20260808 11:44:02.860481 18248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:02.861835 18248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:02.863062 18248 hybrid_clock.cc:648] HybridClock initialized: now 1786189442863022 us; error 50 us; skew 500 ppm
I20260808 11:44:02.863405 18248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:02 dist-test-slave-39l8 krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189442, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:02.872064 18248 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:44:02.873589 18248 webserver.cc:460] Webserver started at http://127.13.158.126:44633/ using document root <none> and password file <none>
I20260808 11:44:02.874315 18248 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:02.874466 18248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:02.874871 18248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:02.877969 18248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "ed8ba6faa0024c4ab3730cb77d03561f"
format_stamp: "Formatted at 2026-08-08 11:44:02 on dist-test-slave-39l8"
server_key: "b80ed2529bb53dd046efd13a0e289aa2"
server_key_iv: "9010b50e768ac125611d8ad022a3fc5e"
server_key_version: "encryptionkey@0"
I20260808 11:44:02.878912 18248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "ed8ba6faa0024c4ab3730cb77d03561f"
format_stamp: "Formatted at 2026-08-08 11:44:02 on dist-test-slave-39l8"
server_key: "b80ed2529bb53dd046efd13a0e289aa2"
server_key_iv: "9010b50e768ac125611d8ad022a3fc5e"
server_key_version: "encryptionkey@0"
I20260808 11:44:02.884145 18248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 11:44:02.887671 18273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:02.889017 18248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:44:02.889217 18248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "ed8ba6faa0024c4ab3730cb77d03561f"
format_stamp: "Formatted at 2026-08-08 11:44:02 on dist-test-slave-39l8"
server_key: "b80ed2529bb53dd046efd13a0e289aa2"
server_key_iv: "9010b50e768ac125611d8ad022a3fc5e"
server_key_version: "encryptionkey@0"
I20260808 11:44:02.889426 18248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:02.913152 18248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:02.917755 18248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:02.918063 18248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:02.936734 18248 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:36035
I20260808 11:44:02.936765 18341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:36035 every 8 connection(s)
I20260808 11:44:02.938120 18248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:02.941962 18342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:02.947842 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 18248
I20260808 11:44:02.948093 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:02.948751 13945 external_mini_cluster.cc:1428] Setting key 9224f878b19f17fa6cc5fb102402b088
I20260808 11:44:02.952986 18342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f: Bootstrap starting.
I20260808 11:44:02.956794 18342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:02.958068 18342 log.cc:824] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:02.961007 18342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f: No bootstrap required, opened a new log
Aug 08 11:44:02 dist-test-slave-39l8 krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:02.970772 18342 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8ba6faa0024c4ab3730cb77d03561f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36035 } }
I20260808 11:44:02.971282 18342 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:02.971400 18342 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed8ba6faa0024c4ab3730cb77d03561f, State: Initialized, Role: FOLLOWER
I20260808 11:44:02.972025 18342 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8ba6faa0024c4ab3730cb77d03561f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36035 } }
I20260808 11:44:02.972290 18342 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:02.972460 18342 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:02.972653 18342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:02.972944 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:02.950635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.13.158.126:36035)
0808 11:44:02.951478 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:02.951490 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:02.951532 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:44:02.953267 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:02.953288 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:02.953668 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:02.953974 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:02.955999 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:02.957420 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:02.959318 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:02.959979 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:02.960191 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:02.964687 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:02.964695 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:02.964710 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:02.964748 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:02.968369 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:02.969276 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:02.969283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:02.969289 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:02.969463 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:02.969879 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:02.969882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:02.969884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:02.970425 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:02.970648 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:02.971172 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 11:44:02.971686 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":375,"threads_started":1}
I20260808 11:44:02.975687 18342 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8ba6faa0024c4ab3730cb77d03561f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36035 } }
I20260808 11:44:02.976263 18342 leader_election.cc:302] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed8ba6faa0024c4ab3730cb77d03561f; no voters:
I20260808 11:44:02.977008 18342 leader_election.cc:288] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:02.977141 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:44:02.977147 18351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:02.979156 18342 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:02.984136 18351 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [term 1 LEADER]: Becoming Leader. State: Replica: ed8ba6faa0024c4ab3730cb77d03561f, State: Running, Role: LEADER
I20260808 11:44:02.984747 18351 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8ba6faa0024c4ab3730cb77d03561f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36035 } }
I20260808 11:44:02.988147 18352 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed8ba6faa0024c4ab3730cb77d03561f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8ba6faa0024c4ab3730cb77d03561f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36035 } } }
I20260808 11:44:02.988583 18352 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:02.989123 18354 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed8ba6faa0024c4ab3730cb77d03561f. Latest consensus state: current_term: 1 leader_uuid: "ed8ba6faa0024c4ab3730cb77d03561f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8ba6faa0024c4ab3730cb77d03561f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36035 } } }
I20260808 11:44:02.989408 18354 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:03.002540 18375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:03.009419 18375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:03.019943 18375 catalog_manager.cc:1347] Generated new cluster ID: 31f2937da72244eabebd71ff4570da67
I20260808 11:44:03.020078 18375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:03.034003 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:03.026293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50080 (local address 127.13.158.126:36035)
0808 11:44:03.026466 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:03.026472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:03.027139 (+ 667us) server_negotiation.cc:407] Connection header received
0808 11:44:03.027365 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:03.027371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:03.027445 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:03.027552 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:03.028583 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:03.029395 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:03.030668 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:03.031158 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:03.031313 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:03.032453 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:03.032458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:03.032462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:03.033084 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:03.033148 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:03.033169 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:03.033370 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:03.033497 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 11:44:03.033589 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":46}
I20260808 11:44:03.075917 18375 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:03.077826 18375 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:03.095295 18375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed8ba6faa0024c4ab3730cb77d03561f: Generated new TSK 0
I20260808 11:44:03.096354 18375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:44:03.110275 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18248
I20260808 11:44:03.111752 18335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:03.205976 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 18248
2026-08-08T11:44:03Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2182 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:03 dist-test-slave-39l8 krb5kdc[18395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:03 dist-test-slave-39l8 krb5kdc[18395](info): set up 2 sockets
Aug 08 11:44:03 dist-test-slave-39l8 krb5kdc[18395](info): commencing operation
krb5kdc: starting...
W20260808 11:44:04.471410 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:04 dist-test-slave-39l8 krb5kdc[18395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:44:04.688359 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36931
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:36931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:05.010777 18413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:05.011076 18413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:05.011260 18413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:05.018450 18413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:05.018551 18413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:05.018659 18413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:05.018800 18413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:05.018849 18413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:05.028575 18413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:36931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:05.029912 18413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:05.031260 18413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:05.043259 18419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:05.043329 18420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:05.043294 18422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:05.044189 18413 server_base.cc:1034] running on GCE node
I20260808 11:44:05.044940 18413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:05.046372 18413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:05.047605 18413 hybrid_clock.cc:648] HybridClock initialized: now 1786189445047563 us; error 49 us; skew 500 ppm
I20260808 11:44:05.047995 18413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:05 dist-test-slave-39l8 krb5kdc[18395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189445, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:05.051321 18413 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:44:05.053333 18413 webserver.cc:460] Webserver started at http://127.13.158.126:35437/ using document root <none> and password file <none>
I20260808 11:44:05.054107 18413 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:05.054267 18413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:05.054602 18413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:05.057864 18413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4"
format_stamp: "Formatted at 2026-08-08 11:44:05 on dist-test-slave-39l8"
server_key: "1c8ea442865c3db3e477fa764f2a5419"
server_key_iv: "67f06542c0535ebabc273e57beea0b21"
server_key_version: "encryptionkey@0"
I20260808 11:44:05.058696 18413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4"
format_stamp: "Formatted at 2026-08-08 11:44:05 on dist-test-slave-39l8"
server_key: "1c8ea442865c3db3e477fa764f2a5419"
server_key_iv: "67f06542c0535ebabc273e57beea0b21"
server_key_version: "encryptionkey@0"
I20260808 11:44:05.064204 18413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:44:05.067859 18431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:05.069201 18413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:44:05.069401 18413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4"
format_stamp: "Formatted at 2026-08-08 11:44:05 on dist-test-slave-39l8"
server_key: "1c8ea442865c3db3e477fa764f2a5419"
server_key_iv: "67f06542c0535ebabc273e57beea0b21"
server_key_version: "encryptionkey@0"
I20260808 11:44:05.069631 18413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:05.103801 18413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:05.107846 18413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:05.108155 18413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:05.127292 18413 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:36931
I20260808 11:44:05.127336 18511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:36931 every 8 connection(s)
I20260808 11:44:05.128660 18413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:05.132349 18512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:05.137563 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 18413
I20260808 11:44:05.137781 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:05.138581 13945 external_mini_cluster.cc:1428] Setting key 36a48e68ac761799ce5dd05c65007e33
I20260808 11:44:05.143790 18512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4: Bootstrap starting.
I20260808 11:44:05.147519 18512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:05.148764 18512 log.cc:824] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:05.152354 18512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4: No bootstrap required, opened a new log
Aug 08 11:44:05 dist-test-slave-39l8 krb5kdc[18395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:05.162943 18512 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36931 } }
I20260808 11:44:05.163393 18512 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:05.163496 18512 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4acefe46b2d4cc5bf3a393ac61b3ad4, State: Initialized, Role: FOLLOWER
I20260808 11:44:05.163954 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:05.140684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.13.158.126:36931)
0808 11:44:05.141582 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:05.141593 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:05.141650 (+ 57us) server_negotiation.cc:407] Connection header received
0808 11:44:05.143354 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:05.143376 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:05.143741 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:05.144099 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:05.145852 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:05.147245 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:05.149068 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:05.150176 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:05.150341 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:05.155106 (+ 4765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:05.155121 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:05.155139 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:05.155189 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:05.159286 (+ 4097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:05.159991 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:05.160000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:05.160008 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:05.160203 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:05.160608 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:05.160612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:05.160616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:05.161392 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:05.161692 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:05.162100 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 11:44:05.162517 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":429,"threads_started":1}
I20260808 11:44:05.164149 18512 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36931 } }
I20260808 11:44:05.164449 18512 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:05.164618 18512 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:05.164834 18512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:05.167266 18512 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36931 } }
I20260808 11:44:05.167754 18512 leader_election.cc:302] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4acefe46b2d4cc5bf3a393ac61b3ad4; no voters:
I20260808 11:44:05.168323 18512 leader_election.cc:288] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:05.168473 18517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:05.169258 18517 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [term 1 LEADER]: Becoming Leader. State: Replica: d4acefe46b2d4cc5bf3a393ac61b3ad4, State: Running, Role: LEADER
I20260808 11:44:05.169816 18517 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36931 } }
I20260808 11:44:05.170409 18512 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:05.172580 18518 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36931 } } }
I20260808 11:44:05.172973 18518 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:05.173532 18520 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4acefe46b2d4cc5bf3a393ac61b3ad4. Latest consensus state: current_term: 1 leader_uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4acefe46b2d4cc5bf3a393ac61b3ad4" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36931 } } }
I20260808 11:44:05.174052 18520 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:05.175796 18528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:05.180953 18528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:05.189857 18528 catalog_manager.cc:1347] Generated new cluster ID: 9cb108dba99440bb9a687109037afb79
I20260808 11:44:05.189970 18528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:05.215595 18528 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:05.216895 18528 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:05.225342 18528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4acefe46b2d4cc5bf3a393ac61b3ad4: Generated new TSK 0
I20260808 11:44:05.226122 18528 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:44:05.291590 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36931
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:05.637338 18552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:05.637719 18552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:05.637838 18552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:05.645231 18552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:05.645345 18552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:05.645457 18552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:05.655529 18552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:05.656790 18552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:05.658025 18552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:05.670919 18560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:05.670966 18562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:05.670989 18558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:05.672125 18552 server_base.cc:1034] running on GCE node
I20260808 11:44:05.672693 18552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:05.673767 18552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:05.674978 18552 hybrid_clock.cc:648] HybridClock initialized: now 1786189445674942 us; error 45 us; skew 500 ppm
I20260808 11:44:05.675288 18552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:05 dist-test-slave-39l8 krb5kdc[18395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189445, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:05.678539 18552 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:44:05.680280 18552 webserver.cc:460] Webserver started at http://127.13.158.65:44297/ using document root <none> and password file <none>
I20260808 11:44:05.681203 18552 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:05.681348 18552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:05.681782 18552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:05.684954 18552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "5bcfcd668e0740d29638d018e23f8dcf"
format_stamp: "Formatted at 2026-08-08 11:44:05 on dist-test-slave-39l8"
server_key: "54b2fc43761296686b6b21f3ff85953b"
server_key_iv: "5eb4f7da4a84a1eff9eb9043acef290a"
server_key_version: "encryptionkey@0"
I20260808 11:44:05.685809 18552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "5bcfcd668e0740d29638d018e23f8dcf"
format_stamp: "Formatted at 2026-08-08 11:44:05 on dist-test-slave-39l8"
server_key: "54b2fc43761296686b6b21f3ff85953b"
server_key_iv: "5eb4f7da4a84a1eff9eb9043acef290a"
server_key_version: "encryptionkey@0"
I20260808 11:44:05.690814 18552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:44:05.694077 18573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:05.695377 18552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:05.695549 18552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "5bcfcd668e0740d29638d018e23f8dcf"
format_stamp: "Formatted at 2026-08-08 11:44:05 on dist-test-slave-39l8"
server_key: "54b2fc43761296686b6b21f3ff85953b"
server_key_iv: "5eb4f7da4a84a1eff9eb9043acef290a"
server_key_version: "encryptionkey@0"
I20260808 11:44:05.695750 18552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:05.709373 18552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:05.713214 18552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:05.713521 18552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:05.714396 18552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:05.715687 18552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:05.715790 18552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:05.715935 18552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:05.716005 18552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:05.748800 18552 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:38783
I20260808 11:44:05.748844 18728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:38783 every 8 connection(s)
I20260808 11:44:05.750104 18552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:05.754571 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 18552
I20260808 11:44:05.754805 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:44:05.755625 13945 external_mini_cluster.cc:1428] Setting key 7e98d6695c38bc4241410bd9d5afbf11
Aug 08 11:44:05 dist-test-slave-39l8 krb5kdc[18395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189445, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:05.773036 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:05.752737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:43477 (local address 127.13.158.126:36931)
0808 11:44:05.753158 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:05.753163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:05.754117 (+ 954us) server_negotiation.cc:407] Connection header received
0808 11:44:05.756625 (+ 2508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:05.756630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:05.756695 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:05.756778 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:05.758649 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:05.759397 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:05.761843 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:05.762236 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:05.762333 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:05.766852 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:05.766857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:05.766860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:05.766893 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:05.769473 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:05.770417 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:05.770421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:05.770423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:05.770598 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:05.771114 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:05.771117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:05.771119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:05.771437 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:05.771578 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:05.772535 (+ 957us) server_negotiation.cc:299] Negotiation successful
0808 11:44:05.772773 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":211,"threads_started":1}
I20260808 11:44:05.774542 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:05.753258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36931 (local address 127.13.158.65:43477)
0808 11:44:05.753936 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:05.753976 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:05.756319 (+ 2343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:05.757047 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:05.757057 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:05.757666 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:05.758449 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:05.758475 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:05.759557 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:05.759565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:05.761674 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:05.761691 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:05.762461 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:05.762470 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:05.762818 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:05.763557 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:05.763591 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:05.766617 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:05.769623 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:05.769635 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:05.769652 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:05.770303 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:05.770754 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:05.770762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:05.770768 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:05.771015 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:05.771576 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:05.771587 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:05.772122 (+ 535us) client_negotiation.cc:769] Sending connection context
0808 11:44:05.772431 (+ 309us) client_negotiation.cc:240] Negotiation successful
0808 11:44:05.772996 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":269,"threads_started":1}
I20260808 11:44:05.776346 18729 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36931
I20260808 11:44:05.776685 18729 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:05.777415 18729 heartbeater.cc:511] Master 127.13.158.126:36931 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
I20260808 11:44:05.779773 18455 ts_manager.cc:194] Registered new tserver with Master: 5bcfcd668e0740d29638d018e23f8dcf (127.13.158.65:38783)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
I20260808 11:44:05.781431 18455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:43477
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:44:05.805837 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36931
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:06.146189 18742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:06.146520 18742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:06.146694 18742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:06.154057 18742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:06.154196 18742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:06.154383 18742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:06.164377 18742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:06.165740 18742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:06.167092 18742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:06.180962 18750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:06.181018 18752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:06.181056 18749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:06.181586 18742 server_base.cc:1034] running on GCE node
I20260808 11:44:06.182263 18742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:06.183375 18742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:06.184607 18742 hybrid_clock.cc:648] HybridClock initialized: now 1786189446184568 us; error 50 us; skew 500 ppm
I20260808 11:44:06.184929 18742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:06.188233 18742 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:44:06.189584 18742 webserver.cc:460] Webserver started at http://127.13.158.66:41975/ using document root <none> and password file <none>
I20260808 11:44:06.190248 18742 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:06.190349 18742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:06.190594 18742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:06.193552 18742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "08c65ab13c2f4f1ba1f3c202d2e42fe1"
format_stamp: "Formatted at 2026-08-08 11:44:06 on dist-test-slave-39l8"
server_key: "7f8225f128e80596bea474c72524f11c"
server_key_iv: "9146db30d8e7219ad4b1eeb97459da86"
server_key_version: "encryptionkey@0"
I20260808 11:44:06.194309 18742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "08c65ab13c2f4f1ba1f3c202d2e42fe1"
format_stamp: "Formatted at 2026-08-08 11:44:06 on dist-test-slave-39l8"
server_key: "7f8225f128e80596bea474c72524f11c"
server_key_iv: "9146db30d8e7219ad4b1eeb97459da86"
server_key_version: "encryptionkey@0"
I20260808 11:44:06.199550 18742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 11:44:06.203183 18760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:06.204600 18742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:44:06.204775 18742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "08c65ab13c2f4f1ba1f3c202d2e42fe1"
format_stamp: "Formatted at 2026-08-08 11:44:06 on dist-test-slave-39l8"
server_key: "7f8225f128e80596bea474c72524f11c"
server_key_iv: "9146db30d8e7219ad4b1eeb97459da86"
server_key_version: "encryptionkey@0"
I20260808 11:44:06.204989 18742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:06.232838 18742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:06.236863 18742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:06.237179 18742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:06.238157 18742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:06.239748 18742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:06.239873 18742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:06.240031 18742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:06.240083 18742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:06.275882 18742 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:41603
I20260808 11:44:06.275924 18924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:41603 every 8 connection(s)
I20260808 11:44:06.277256 18742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:06.287632 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 18742
I20260808 11:44:06.287814 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:06.288401 13945 external_mini_cluster.cc:1428] Setting key 55a80fdb02c22fbc948e5eed0f0edb36
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:06.297617 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:06.279795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:60399 (local address 127.13.158.126:36931)
0808 11:44:06.280224 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:06.280229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:06.281260 (+ 1031us) server_negotiation.cc:407] Connection header received
0808 11:44:06.283145 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:06.283151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:06.283240 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:06.283362 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:06.284830 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.285543 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.287194 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.287860 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.288006 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:06.291729 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:06.291734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:06.291737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:06.291770 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:06.294446 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.295250 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.295255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.295257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.295400 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.295869 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.295879 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.295880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.296152 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:06.296312 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:06.297008 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 11:44:06.297252 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":201,"threads_started":1}
I20260808 11:44:06.298553 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:06.280373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36931 (local address 127.13.158.66:60399)
0808 11:44:06.281090 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:06.281137 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:06.282874 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:06.283589 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:06.283596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:06.284090 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:06.284655 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:06.284672 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.285719 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.285723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:06.287008 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:06.287021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.288082 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.288088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:06.288293 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:06.289022 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:06.289045 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:06.291500 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:06.294592 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:06.294601 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:06.294624 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:06.295111 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:06.295520 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:06.295525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:06.295529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:06.295736 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:06.296287 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:06.296295 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:06.296634 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 11:44:06.296899 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 11:44:06.297414 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":276,"threads_started":1}
I20260808 11:44:06.300094 18925 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36931
I20260808 11:44:06.300442 18925 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:06.301314 18925 heartbeater.cc:511] Master 127.13.158.126:36931 requested a full tablet report, sending...
I20260808 11:44:06.302919 18455 ts_manager.cc:194] Registered new tserver with Master: 08c65ab13c2f4f1ba1f3c202d2e42fe1 (127.13.158.66:41603)
I20260808 11:44:06.304219 18455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:60399
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:44:06.336252 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36931
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:06.663128 18934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:06.663400 18934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:06.663578 18934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:06.670488 18934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:06.670590 18934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:06.670749 18934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:06.680209 18934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:06.681422 18934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:06.682641 18934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:06.696408 18943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:06.696368 18945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:06.696822 18942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:06.697266 18934 server_base.cc:1034] running on GCE node
I20260808 11:44:06.697847 18934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:06.698782 18934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:06.699999 18934 hybrid_clock.cc:648] HybridClock initialized: now 1786189446699944 us; error 56 us; skew 500 ppm
I20260808 11:44:06.700304 18934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:06.703516 18934 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:44:06.704926 18934 webserver.cc:460] Webserver started at http://127.13.158.67:45149/ using document root <none> and password file <none>
I20260808 11:44:06.705575 18934 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:06.705729 18934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:06.706029 18934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:06.709089 18934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "d400b179063140508562bd12561441d0"
format_stamp: "Formatted at 2026-08-08 11:44:06 on dist-test-slave-39l8"
server_key: "9a243a6605774f943e8c175853af462d"
server_key_iv: "49650d397bd2bf0519720726243c7a20"
server_key_version: "encryptionkey@0"
I20260808 11:44:06.709919 18934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "d400b179063140508562bd12561441d0"
format_stamp: "Formatted at 2026-08-08 11:44:06 on dist-test-slave-39l8"
server_key: "9a243a6605774f943e8c175853af462d"
server_key_iv: "49650d397bd2bf0519720726243c7a20"
server_key_version: "encryptionkey@0"
I20260808 11:44:06.714865 18934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260808 11:44:06.718012 18954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:06.719360 18934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:06.719533 18934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "d400b179063140508562bd12561441d0"
format_stamp: "Formatted at 2026-08-08 11:44:06 on dist-test-slave-39l8"
server_key: "9a243a6605774f943e8c175853af462d"
server_key_iv: "49650d397bd2bf0519720726243c7a20"
server_key_version: "encryptionkey@0"
I20260808 11:44:06.719753 18934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:06.736399 18934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:06.740795 18934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:06.741088 18934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:06.742036 18934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:06.743378 18934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:06.743480 18934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:06.743629 18934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:06.743701 18934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:06.777383 18934 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:33451
I20260808 11:44:06.777420 19118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:33451 every 8 connection(s)
I20260808 11:44:06.778637 18934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:06.785096 18729 heartbeater.cc:503] Master 127.13.158.126:36931 was elected leader, sending a full tablet report...
I20260808 11:44:06.785706 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 18934
I20260808 11:44:06.785893 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:06.786438 13945 external_mini_cluster.cc:1428] Setting key b00e104c2f5d65be14a63d7279856c07
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:06.798257 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:06.781275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:43483 (local address 127.13.158.126:36931)
0808 11:44:06.781449 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:06.781453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:06.782472 (+ 1019us) server_negotiation.cc:407] Connection header received
0808 11:44:06.784275 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:06.784292 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:06.784376 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:06.784463 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:06.786263 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.787283 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.789063 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.789597 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.789693 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:06.793036 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:06.793040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:06.793043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:06.793074 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:06.795726 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.796362 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.796366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.796368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.796479 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.796854 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.796857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.796859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.797116 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:06.797257 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:06.797849 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 11:44:06.798004 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:44:06.799371 19122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:06.781627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36931 (local address 127.13.158.67:43483)
0808 11:44:06.782304 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:06.782344 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:06.783995 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:06.784685 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:06.784692 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:06.785221 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:06.786052 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:06.786076 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.787441 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.787446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:06.788925 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:06.788936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.789785 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.789790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:06.789987 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:06.790602 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:06.790626 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:06.792854 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:06.795848 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:06.795854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:06.795865 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:06.796268 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:06.796570 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:06.796573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:06.796575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:06.796761 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:06.797248 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:06.797254 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:06.797550 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 11:44:06.797770 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 11:44:06.798156 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":266,"threads_started":1}
I20260808 11:44:06.801069 19119 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36931
I20260808 11:44:06.801410 19119 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:06.802273 19119 heartbeater.cc:511] Master 127.13.158.126:36931 requested a full tablet report, sending...
I20260808 11:44:06.803653 18452 ts_manager.cc:194] Registered new tserver with Master: d400b179063140508562bd12561441d0 (127.13.158.67:33451)
I20260808 11:44:06.804505 18452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:43483
I20260808 11:44:06.814054 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:06.865267 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:06.851741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50080 (local address 127.13.158.126:36931)
0808 11:44:06.851910 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:06.851914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:06.852538 (+ 624us) server_negotiation.cc:407] Connection header received
0808 11:44:06.852696 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:06.852700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:06.852789 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:06.852892 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:06.854256 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.855010 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.856188 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.856541 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.856623 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:06.860159 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:06.860164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:06.860167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:06.860199 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:06.862674 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.863336 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.863339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.863342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.863463 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.863852 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.863858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.863862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.864170 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:06.864442 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:06.864715 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 11:44:06.864943 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 11:44:06.865830 18452 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50080
I20260808 11:44:06.867985 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18552
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:44:06.884339 19137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:06.869045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.13.158.65:38783)
0808 11:44:06.869687 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:06.869694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:06.869724 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:44:06.869780 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:06.869786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:06.869986 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:06.870107 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:06.871339 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.872445 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.873780 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.874631 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.874802 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:06.878837 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:06.878848 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:06.878861 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:06.878898 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:06.881756 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.882382 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.882389 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.882391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.882505 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.882992 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.882995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.882997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.883308 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:06.883521 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:06.883758 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 11:44:06.883976 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":342,"threads_started":1}
W20260808 11:44:06.885571 18723 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41854
W20260808 11:44:06.886662 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18552: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:44:06.886917 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 18552
I20260808 11:44:06.910995 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18742
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:44:06.926339 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:06.911909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.13.158.66:41603)
0808 11:44:06.912560 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:06.912568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:06.912595 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:44:06.912665 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:06.912672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:06.912885 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:06.913018 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:06.914212 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.915261 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.916448 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.917015 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.917111 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:06.921197 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:06.921209 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:06.921223 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:06.921253 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:06.923988 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.924599 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.924606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.924609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.924729 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.925056 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.925059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.925060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.925408 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:06.925633 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:06.925848 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 11:44:06.926037 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":354,"threads_started":1}
W20260808 11:44:06.927657 18920 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43076
W20260808 11:44:06.928794 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18742: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:44:06.928962 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 18742
I20260808 11:44:06.949281 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18934
Aug 08 11:44:06 dist-test-slave-39l8 krb5kdc[18395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189446, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:44:06.964862 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:06.950347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33342 (local address 127.13.158.67:33451)
0808 11:44:06.951143 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:06.951150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:06.951180 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:44:06.951241 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:06.951247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:06.951440 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:06.951567 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:06.952812 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.953832 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.955051 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:06.955591 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:06.955685 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:06.959773 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:06.959783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:06.959795 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:06.959842 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:06.962355 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.963133 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.963139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.963141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.963254 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:06.963556 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:06.963559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:06.963560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:06.963848 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:06.964055 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:06.964300 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 11:44:06.964510 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":438,"threads_started":1}
W20260808 11:44:06.966169 19113 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33342
W20260808 11:44:06.967398 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18934: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:44:06.967586 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 18934
I20260808 11:44:06.992122 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 18413
I20260808 11:44:06.993402 18499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:07.094856 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 18413
2026-08-08T11:44:07Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3894 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:07 dist-test-slave-39l8 krb5kdc[19146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:07 dist-test-slave-39l8 krb5kdc[19146](info): set up 2 sockets
Aug 08 11:44:07 dist-test-slave-39l8 krb5kdc[19146](info): commencing operation
krb5kdc: starting...
W20260808 11:44:08.371995 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.008s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:08 dist-test-slave-39l8 krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:44:08.573786 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43335
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:41933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:43335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:08.898854 19167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:08.899189 19167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:08.899344 19167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:08.906355 19167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:08.906502 19167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:08.906629 19167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:08.906755 19167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:08.906836 19167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:08.916893 19167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:41933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:43335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:08.918257 19167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:08.919642 19167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:08.931639 19173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:08.931687 19176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:08.931820 19178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:08.932799 19167 server_base.cc:1034] running on GCE node
I20260808 11:44:08.933617 19167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:08.934942 19167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:08.936173 19167 hybrid_clock.cc:648] HybridClock initialized: now 1786189448936135 us; error 46 us; skew 500 ppm
I20260808 11:44:08.936502 19167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:08 dist-test-slave-39l8 krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189448, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:08.944144 19167 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:44:08.945652 19167 webserver.cc:460] Webserver started at http://127.13.158.126:38637/ using document root <none> and password file <none>
I20260808 11:44:08.946352 19167 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:08.946491 19167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:08.946854 19167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:08.949797 19167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "6e7088ee75ef42909fb80392eb127d93"
format_stamp: "Formatted at 2026-08-08 11:44:08 on dist-test-slave-39l8"
server_key: "cb1ad5f5ebb67797867ee87be1cebbdf"
server_key_iv: "eb459d7b21b818648468fdbfbd99377a"
server_key_version: "encryptionkey@0"
I20260808 11:44:08.950590 19167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "6e7088ee75ef42909fb80392eb127d93"
format_stamp: "Formatted at 2026-08-08 11:44:08 on dist-test-slave-39l8"
server_key: "cb1ad5f5ebb67797867ee87be1cebbdf"
server_key_iv: "eb459d7b21b818648468fdbfbd99377a"
server_key_version: "encryptionkey@0"
I20260808 11:44:08.955664 19167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:44:08.958912 19189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:08.960124 19167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:44:08.960314 19167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "6e7088ee75ef42909fb80392eb127d93"
format_stamp: "Formatted at 2026-08-08 11:44:08 on dist-test-slave-39l8"
server_key: "cb1ad5f5ebb67797867ee87be1cebbdf"
server_key_iv: "eb459d7b21b818648468fdbfbd99377a"
server_key_version: "encryptionkey@0"
I20260808 11:44:08.960508 19167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:08.983239 19167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:08.987423 19167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:08.987730 19167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:09.006143 19167 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:43335
I20260808 11:44:09.006155 19252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:43335 every 8 connection(s)
I20260808 11:44:09.007448 19167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:09.011018 19253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:09.012374 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 19167
I20260808 11:44:09.012646 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:09.013310 13945 external_mini_cluster.cc:1428] Setting key e130ffdfc19c5dbdac54c251cbe491f5
I20260808 11:44:09.025079 19253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93: Bootstrap starting.
I20260808 11:44:09.028250 19253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93: Neither blocks nor log segments found. Creating new log.
Aug 08 11:44:09 dist-test-slave-39l8 krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:09.029325 19253 log.cc:824] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:09.032886 19253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93: No bootstrap required, opened a new log
I20260808 11:44:09.038002 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:09.015346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.13.158.126:43335)
0808 11:44:09.016239 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:09.016259 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:09.016317 (+ 58us) server_negotiation.cc:407] Connection header received
0808 11:44:09.017969 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:09.017999 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:09.018473 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:09.018881 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:09.020304 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:09.022443 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:09.023783 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:09.024818 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:09.024994 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:09.029341 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:09.029373 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:09.029392 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:09.029444 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:09.033586 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:09.034251 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:09.034260 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:09.034268 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:09.034462 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:09.034922 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:09.034927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:09.034930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:09.035592 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:09.035926 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:09.036250 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 11:44:09.036687 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":369,"threads_started":1}
I20260808 11:44:09.042479 19253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7088ee75ef42909fb80392eb127d93" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43335 } }
I20260808 11:44:09.042893 19253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:09.043005 19253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7088ee75ef42909fb80392eb127d93, State: Initialized, Role: FOLLOWER
I20260808 11:44:09.043493 19253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7088ee75ef42909fb80392eb127d93" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43335 } }
I20260808 11:44:09.043716 19253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:09.043870 19253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:09.044037 19253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:09.046108 19253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7088ee75ef42909fb80392eb127d93" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43335 } }
I20260808 11:44:09.046572 19253 leader_election.cc:302] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e7088ee75ef42909fb80392eb127d93; no voters:
I20260808 11:44:09.047271 19253 leader_election.cc:288] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:09.047405 19260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:09.048063 19260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [term 1 LEADER]: Becoming Leader. State: Replica: 6e7088ee75ef42909fb80392eb127d93, State: Running, Role: LEADER
I20260808 11:44:09.048596 19260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7088ee75ef42909fb80392eb127d93" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43335 } }
I20260808 11:44:09.049439 19253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:09.051666 19261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e7088ee75ef42909fb80392eb127d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7088ee75ef42909fb80392eb127d93" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43335 } } }
I20260808 11:44:09.052153 19261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:09.051715 19263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e7088ee75ef42909fb80392eb127d93. Latest consensus state: current_term: 1 leader_uuid: "6e7088ee75ef42909fb80392eb127d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7088ee75ef42909fb80392eb127d93" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43335 } } }
I20260808 11:44:09.053336 19263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:09.054374 19268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:09.059347 19268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:09.067636 19268 catalog_manager.cc:1347] Generated new cluster ID: 22bd3f47fc6341d5b9ea8dc722bcdfa6
I20260808 11:44:09.067740 19268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:09.085496 19268 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:09.086763 19268 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:09.093854 19268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e7088ee75ef42909fb80392eb127d93: Generated new TSK 0
I20260808 11:44:09.094640 19268 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:44:09.149893 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43335
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:41933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:09.482170 19297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:09.482436 19297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:09.482527 19297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:09.489480 19297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:09.489591 19297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:09.489740 19297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:09.499184 19297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:41933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:09.500435 19297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:09.501722 19297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:09.514919 19307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:09.515004 19305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:09.515008 19309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:09.515858 19297 server_base.cc:1034] running on GCE node
I20260808 11:44:09.516434 19297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:09.517333 19297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:09.518532 19297 hybrid_clock.cc:648] HybridClock initialized: now 1786189449518496 us; error 30 us; skew 500 ppm
I20260808 11:44:09.518909 19297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:09 dist-test-slave-39l8 krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189449, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:09.522024 19297 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:44:09.523582 19297 webserver.cc:460] Webserver started at http://127.13.158.65:39143/ using document root <none> and password file <none>
I20260808 11:44:09.524304 19297 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:09.524454 19297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:09.524777 19297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:09.527632 19297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "8dc6e9e98f024902952d4a6613d80d0c"
format_stamp: "Formatted at 2026-08-08 11:44:09 on dist-test-slave-39l8"
server_key: "ef0614bd9724ab8c664e26fa51e92deb"
server_key_iv: "a7ba9e6b3059ac1c4ce2ec24fd01ec6c"
server_key_version: "encryptionkey@0"
I20260808 11:44:09.528450 19297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "8dc6e9e98f024902952d4a6613d80d0c"
format_stamp: "Formatted at 2026-08-08 11:44:09 on dist-test-slave-39l8"
server_key: "ef0614bd9724ab8c664e26fa51e92deb"
server_key_iv: "a7ba9e6b3059ac1c4ce2ec24fd01ec6c"
server_key_version: "encryptionkey@0"
I20260808 11:44:09.533520 19297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:44:09.536828 19319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:09.538014 19297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:44:09.538195 19297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "8dc6e9e98f024902952d4a6613d80d0c"
format_stamp: "Formatted at 2026-08-08 11:44:09 on dist-test-slave-39l8"
server_key: "ef0614bd9724ab8c664e26fa51e92deb"
server_key_iv: "a7ba9e6b3059ac1c4ce2ec24fd01ec6c"
server_key_version: "encryptionkey@0"
I20260808 11:44:09.538388 19297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:09.588410 19297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:09.593035 19297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:09.593338 19297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:09.594368 19297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:09.595821 19297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:09.595927 19297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:09.596066 19297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:09.596117 19297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:09.629804 19297 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:46247
I20260808 11:44:09.629886 19478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:46247 every 8 connection(s)
I20260808 11:44:09.631202 19297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:09.632778 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 19297
I20260808 11:44:09.633031 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:44:09.633769 13945 external_mini_cluster.cc:1428] Setting key c52c3e97bd0e81a64c640cd07bc307c1
Aug 08 11:44:09 dist-test-slave-39l8 krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189449, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:09.651623 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:09.633812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:54483 (local address 127.13.158.126:43335)
0808 11:44:09.634681 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:09.634687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:09.635810 (+ 1123us) server_negotiation.cc:407] Connection header received
0808 11:44:09.637406 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:09.637411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:09.637494 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:09.637618 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:09.639114 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:09.639796 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:09.641357 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:09.641805 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:09.641949 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:09.645427 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:09.645435 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:09.645444 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:09.645475 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:09.648188 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:09.648893 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:09.648898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:09.648903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:09.649019 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:09.649727 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:09.649732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:09.649734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:09.650055 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:09.650211 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:09.651046 (+ 835us) server_negotiation.cc:299] Negotiation successful
0808 11:44:09.651262 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":607,"threads_started":1}
I20260808 11:44:09.652521 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:09.634465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43335 (local address 127.13.158.65:54483)
0808 11:44:09.635415 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:09.635461 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:09.637141 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:09.637848 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:09.637856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:09.638359 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:09.638951 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:09.638970 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:09.639960 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:09.639964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:09.641197 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:09.641217 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:09.642072 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:09.642082 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:09.642347 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:09.643099 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:09.643120 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:09.645220 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:09.648330 (+ 3110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:09.648339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:09.648352 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:09.648766 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:09.649345 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:09.649350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:09.649355 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:09.649587 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:09.650185 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:09.650193 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:09.650587 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 11:44:09.650898 (+ 311us) client_negotiation.cc:240] Negotiation successful
0808 11:44:09.651414 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":418,"threads_started":1}
I20260808 11:44:09.653978 19480 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43335
I20260808 11:44:09.654313 19480 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:09.655057 19480 heartbeater.cc:511] Master 127.13.158.126:43335 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
I20260808 11:44:09.657317 19211 ts_manager.cc:194] Registered new tserver with Master: 8dc6e9e98f024902952d4a6613d80d0c (127.13.158.65:46247)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
I20260808 11:44:09.658947 19211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:54483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:44:09.681648 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43335
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:41933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:10.010434 19488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:10.010697 19488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:10.010927 19488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:10.021101 19488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:10.021306 19488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:10.021517 19488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:10.033985 19488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:41933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:10.035511 19488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:10.036828 19488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:10.050165 19494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:10.050230 19495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:10.050261 19497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:10.051110 19488 server_base.cc:1034] running on GCE node
I20260808 11:44:10.051625 19488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:10.052498 19488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:10.053700 19488 hybrid_clock.cc:648] HybridClock initialized: now 1786189450053648 us; error 50 us; skew 500 ppm
I20260808 11:44:10.054054 19488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:10 dist-test-slave-39l8 krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:10.057489 19488 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:44:10.058987 19488 webserver.cc:460] Webserver started at http://127.13.158.66:39181/ using document root <none> and password file <none>
I20260808 11:44:10.059679 19488 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:10.059826 19488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:10.060144 19488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:10.062994 19488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "44fbb6e541494c5298ccda0e3c814c2b"
format_stamp: "Formatted at 2026-08-08 11:44:10 on dist-test-slave-39l8"
server_key: "cafc9e9f78d6a333bf1085cb6397ded5"
server_key_iv: "ed738306d16f2a33ffaa18575e0c3ffb"
server_key_version: "encryptionkey@0"
I20260808 11:44:10.063879 19488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "44fbb6e541494c5298ccda0e3c814c2b"
format_stamp: "Formatted at 2026-08-08 11:44:10 on dist-test-slave-39l8"
server_key: "cafc9e9f78d6a333bf1085cb6397ded5"
server_key_iv: "ed738306d16f2a33ffaa18575e0c3ffb"
server_key_version: "encryptionkey@0"
I20260808 11:44:10.068955 19488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:44:10.072412 19508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:10.073700 19488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:44:10.073879 19488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "44fbb6e541494c5298ccda0e3c814c2b"
format_stamp: "Formatted at 2026-08-08 11:44:10 on dist-test-slave-39l8"
server_key: "cafc9e9f78d6a333bf1085cb6397ded5"
server_key_iv: "ed738306d16f2a33ffaa18575e0c3ffb"
server_key_version: "encryptionkey@0"
I20260808 11:44:10.074084 19488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:10.109665 19488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:10.113703 19488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:10.114008 19488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:10.115041 19488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:10.116377 19488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:10.116492 19488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:10.116663 19488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:10.116739 19488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:10.150606 19488 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:43943
I20260808 11:44:10.150682 19673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:43943 every 8 connection(s)
I20260808 11:44:10.151993 19488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:10.161279 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 19488
I20260808 11:44:10.161477 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:10.161993 13945 external_mini_cluster.cc:1428] Setting key e0d6b4b552fc8919953aafe149bdf4ff
Aug 08 11:44:10 dist-test-slave-39l8 krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:10.172315 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:10.154555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:60367 (local address 127.13.158.126:43335)
0808 11:44:10.155059 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:10.155066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:10.156112 (+ 1046us) server_negotiation.cc:407] Connection header received
0808 11:44:10.157906 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:10.157910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:10.157976 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:10.158080 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:10.159544 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.160258 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.162313 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.162913 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.163053 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:10.166556 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:10.166561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:10.166564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:10.166595 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:10.169254 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.169952 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.169960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.169966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.170110 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.170543 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.170549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.170552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.170876 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:10.171031 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:10.171698 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 11:44:10.171909 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":265,"threads_started":1}
I20260808 11:44:10.173216 19680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:10.155150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43335 (local address 127.13.158.66:60367)
0808 11:44:10.155949 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:10.155994 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:10.157670 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:10.158273 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:10.158280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:10.158862 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:10.159394 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:10.159410 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.160389 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.160393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:10.162173 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:10.162189 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.163151 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.163161 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:10.163390 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:10.164147 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:10.164175 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:10.166330 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:10.169399 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:10.169408 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:10.169422 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:10.169831 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:10.170231 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:10.170236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:10.170239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:10.170426 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:10.171005 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:10.171013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:10.171332 (+ 319us) client_negotiation.cc:769] Sending connection context
0808 11:44:10.171608 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 11:44:10.172125 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":346,"threads_started":1}
I20260808 11:44:10.174763 19675 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43335
I20260808 11:44:10.175076 19675 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:10.175776 19675 heartbeater.cc:511] Master 127.13.158.126:43335 requested a full tablet report, sending...
I20260808 11:44:10.177342 19211 ts_manager.cc:194] Registered new tserver with Master: 44fbb6e541494c5298ccda0e3c814c2b (127.13.158.66:43943)
I20260808 11:44:10.178666 19211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:60367
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:44:10.210002 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43335
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:41933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:10.538864 19683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:10.539163 19683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:10.539316 19683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:10.546818 19683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:10.546963 19683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:10.547161 19683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:10.557096 19683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:41933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:10.558269 19683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:10.559612 19683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:10.572336 19692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:10.572427 19696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:10.572840 19690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:10.573182 19683 server_base.cc:1034] running on GCE node
I20260808 11:44:10.573760 19683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:10.574944 19683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:10.576220 19683 hybrid_clock.cc:648] HybridClock initialized: now 1786189450576157 us; error 50 us; skew 500 ppm
I20260808 11:44:10.576539 19683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:10 dist-test-slave-39l8 krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:10.580014 19683 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:44:10.581382 19683 webserver.cc:460] Webserver started at http://127.13.158.67:46619/ using document root <none> and password file <none>
I20260808 11:44:10.582036 19683 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:10.582158 19683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:10.582412 19683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:10.585408 19683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "eff046f9af134932aeb8407ab45d7baf"
format_stamp: "Formatted at 2026-08-08 11:44:10 on dist-test-slave-39l8"
server_key: "8f05fdc632fd70d079f9c2dae62152db"
server_key_iv: "47e92d72b5beada37c45e2dd4b1e4775"
server_key_version: "encryptionkey@0"
I20260808 11:44:10.586128 19683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "eff046f9af134932aeb8407ab45d7baf"
format_stamp: "Formatted at 2026-08-08 11:44:10 on dist-test-slave-39l8"
server_key: "8f05fdc632fd70d079f9c2dae62152db"
server_key_iv: "47e92d72b5beada37c45e2dd4b1e4775"
server_key_version: "encryptionkey@0"
I20260808 11:44:10.591207 19683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 11:44:10.594662 19708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:10.596045 19683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:44:10.596195 19683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "eff046f9af134932aeb8407ab45d7baf"
format_stamp: "Formatted at 2026-08-08 11:44:10 on dist-test-slave-39l8"
server_key: "8f05fdc632fd70d079f9c2dae62152db"
server_key_iv: "47e92d72b5beada37c45e2dd4b1e4775"
server_key_version: "encryptionkey@0"
I20260808 11:44:10.596364 19683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:10.636024 19683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:10.640107 19683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:10.640373 19683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:10.641258 19683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:10.642642 19683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:10.642776 19683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:10.642879 19683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:10.642926 19683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:10.661808 19480 heartbeater.cc:503] Master 127.13.158.126:43335 was elected leader, sending a full tablet report...
I20260808 11:44:10.677181 19683 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:38951
I20260808 11:44:10.677258 19885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:38951 every 8 connection(s)
I20260808 11:44:10.678558 19683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:10.681015 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 19683
I20260808 11:44:10.681239 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:10.681777 13945 external_mini_cluster.cc:1428] Setting key a52fd7ec18d75afa53d3e8f0cc0b78f1
Aug 08 11:44:10 dist-test-slave-39l8 krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:10.698496 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:10.681235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:51851 (local address 127.13.158.126:43335)
0808 11:44:10.681770 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:10.681778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:10.682821 (+ 1043us) server_negotiation.cc:407] Connection header received
0808 11:44:10.685134 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:10.685139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:10.685229 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:10.685345 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:10.686843 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.687642 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.689106 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.689486 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.689618 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:10.693008 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:10.693014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:10.693017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:10.693047 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:10.695503 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.696203 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.696213 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.696215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.696325 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.696731 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.696736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.696740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.696996 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:10.697158 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:10.697984 (+ 826us) server_negotiation.cc:299] Negotiation successful
0808 11:44:10.698212 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":257,"threads_started":1}
I20260808 11:44:10.699851 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:10.681823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43335 (local address 127.13.158.67:51851)
0808 11:44:10.682650 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:10.682733 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:10.684843 (+ 2110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:10.685575 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:10.685583 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:10.686108 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:10.686664 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:10.686679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.687799 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.687804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:10.688960 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:10.688972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.689698 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.689705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:10.689942 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:10.690634 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:10.690655 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:10.692817 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:10.695644 (+ 2827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:10.695652 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:10.695665 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:10.696074 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:10.696433 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:10.696439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:10.696442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:10.696611 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:10.697132 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:10.697139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:10.697488 (+ 349us) client_negotiation.cc:769] Sending connection context
0808 11:44:10.697837 (+ 349us) client_negotiation.cc:240] Negotiation successful
0808 11:44:10.698325 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":337,"threads_started":1}
I20260808 11:44:10.701412 19886 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43335
I20260808 11:44:10.701655 19886 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:10.702272 19886 heartbeater.cc:511] Master 127.13.158.126:43335 requested a full tablet report, sending...
I20260808 11:44:10.703534 19211 ts_manager.cc:194] Registered new tserver with Master: eff046f9af134932aeb8407ab45d7baf (127.13.158.67:38951)
I20260808 11:44:10.704427 19211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:51851
I20260808 11:44:10.708899 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:44:10 dist-test-slave-39l8 krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:44:10 dist-test-slave-39l8 krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:44:10.759663 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:10.744982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59606 (local address 127.13.158.65:46247)
0808 11:44:10.745560 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:10.745567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:10.745594 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:44:10.745655 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:10.745661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:10.745895 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:10.746040 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:10.747471 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.748412 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.749835 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.750455 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.750553 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:10.754310 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:10.754320 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:10.754334 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:10.754362 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:10.757280 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.757881 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.757887 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.757889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.757999 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.758315 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.758317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.758319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.758603 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:10.758892 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:10.759070 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 11:44:10.759256 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":289,"threads_started":1}
Aug 08 11:44:10 dist-test-slave-39l8 krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:10.777072 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:10.764243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.13.158.126:43335)
0808 11:44:10.764453 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:10.764458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:10.764556 (+ 98us) server_negotiation.cc:407] Connection header received
0808 11:44:10.764694 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:10.764698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:10.764813 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:10.764888 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:10.766120 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.766900 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.768032 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.768405 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.768491 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:10.771977 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:10.771982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:10.771985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:10.772025 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:10.774587 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.775317 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.775321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.775323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.775428 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.775788 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.775790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.775792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.776084 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:10.776259 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:10.776510 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 11:44:10.776658 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 11:44:10.777634 19211 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57404
I20260808 11:44:10.778540 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 19297
I20260808 11:44:10.790138 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:10.779505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.13.158.65:46247)
0808 11:44:10.779698 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:10.779704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:10.779817 (+ 113us) server_negotiation.cc:407] Connection header received
0808 11:44:10.779927 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:10.779931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:10.779996 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:10.780068 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:10.781167 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.781857 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.783023 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.783405 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.783496 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:10.785473 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:10.785477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:10.785480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:10.785508 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:10.788131 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.788655 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.788658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.788660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.788777 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.789072 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.789075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.789077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.789283 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:10.789445 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:10.789651 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 11:44:10.789806 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:44:10.790799 19472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:10.872288 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 19297
I20260808 11:44:10.895380 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 19488
Aug 08 11:44:10 dist-test-slave-39l8 krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:44:10.911388 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:10.896378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.13.158.66:43943)
0808 11:44:10.896917 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:10.896925 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:10.896950 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:44:10.897013 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:10.897020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:10.897252 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:10.897432 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:10.898650 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.899820 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.901198 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:10.901725 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:10.901862 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:10.905882 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:10.905893 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:10.905908 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:10.905939 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:10.908569 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.909227 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.909234 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.909236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.909356 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:10.909741 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:10.909746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:10.909750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:10.910073 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:10.910429 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:10.910758 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 11:44:10.911024 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":291,"threads_started":1}
I20260808 11:44:10.912659 19666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:10.994845 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 19488
I20260808 11:44:11.019088 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 19683
Aug 08 11:44:11 dist-test-slave-39l8 krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:44:11.035137 19906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:11.020205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.13.158.67:38951)
0808 11:44:11.020853 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:11.020860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:11.020890 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:44:11.020948 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:11.020954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:11.021162 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:11.021288 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:11.022473 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:11.023509 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:11.024788 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:11.025311 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:11.025403 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:11.029745 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:11.029756 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:11.029775 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:11.029810 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:11.032716 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:11.033314 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:11.033320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:11.033322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:11.033429 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:11.033748 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:11.033751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:11.033752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:11.034054 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:11.034280 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:11.034501 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 11:44:11.034685 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":340,"threads_started":1}
I20260808 11:44:11.036523 19876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:11.119567 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 19683
I20260808 11:44:11.140360 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 19167
I20260808 11:44:11.141608 19246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:11.243620 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 19167
2026-08-08T11:44:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4149 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:11 dist-test-slave-39l8 krb5kdc[19912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:11 dist-test-slave-39l8 krb5kdc[19912](info): set up 2 sockets
Aug 08 11:44:11 dist-test-slave-39l8 krb5kdc[19912](info): commencing operation
krb5kdc: starting...
W20260808 11:44:12.503684 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:12 dist-test-slave-39l8 krb5kdc[19912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:44:12.713665 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44631
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:44631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:13.035965 19930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:13.036240 19930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:13.036381 19930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:13.043498 19930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:13.043610 19930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:13.043694 19930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:13.043803 19930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:13.043895 19930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:13.053553 19930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:44631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:13.054836 19930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:13.056216 19930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:13.067718 19941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:13.067778 19940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:13.067755 19943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:13.068856 19930 server_base.cc:1034] running on GCE node
I20260808 11:44:13.069600 19930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:13.071044 19930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:13.072279 19930 hybrid_clock.cc:648] HybridClock initialized: now 1786189453072209 us; error 27 us; skew 500 ppm
I20260808 11:44:13.072652 19930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:13 dist-test-slave-39l8 krb5kdc[19912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:13.075917 19930 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:44:13.077208 19930 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:13.077353 19930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:13.077702 19930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:13.080695 19930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "6a23b34ecd304708a67054698b0df649"
format_stamp: "Formatted at 2026-08-08 11:44:13 on dist-test-slave-39l8"
server_key: "768cc6e31cf6ad49523e2e6052086247"
server_key_iv: "ba010c9c66b747d81c5764d014c7fec6"
server_key_version: "encryptionkey@0"
I20260808 11:44:13.081494 19930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "6a23b34ecd304708a67054698b0df649"
format_stamp: "Formatted at 2026-08-08 11:44:13 on dist-test-slave-39l8"
server_key: "768cc6e31cf6ad49523e2e6052086247"
server_key_iv: "ba010c9c66b747d81c5764d014c7fec6"
server_key_version: "encryptionkey@0"
I20260808 11:44:13.086594 19930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 11:44:13.089920 19952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:13.091176 19930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:13.091354 19930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "6a23b34ecd304708a67054698b0df649"
format_stamp: "Formatted at 2026-08-08 11:44:13 on dist-test-slave-39l8"
server_key: "768cc6e31cf6ad49523e2e6052086247"
server_key_iv: "ba010c9c66b747d81c5764d014c7fec6"
server_key_version: "encryptionkey@0"
I20260808 11:44:13.091564 19930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:13.108055 19930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:13.112131 19930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:13.112413 19930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:13.130188 19930 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:44631
I20260808 11:44:13.130201 20036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:44631 every 8 connection(s)
I20260808 11:44:13.131181 19930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:13.133569 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 19930
I20260808 11:44:13.133817 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:13.134491 13945 external_mini_cluster.cc:1428] Setting key 5ca6ecc936dc87637814044a7822486d
I20260808 11:44:13.135176 20038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:13.149263 20038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649: Bootstrap starting.
Aug 08 11:44:13 dist-test-slave-39l8 krb5kdc[19912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:13.153267 20038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:13.154642 20038 log.cc:824] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:13.158187 20038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649: No bootstrap required, opened a new log
I20260808 11:44:13.159478 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:13.136532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34390 (local address 127.13.158.126:44631)
0808 11:44:13.137366 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:13.137381 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:13.137429 (+ 48us) server_negotiation.cc:407] Connection header received
0808 11:44:13.140006 (+ 2577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:13.140037 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:13.140555 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:13.140975 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:13.142336 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:13.144314 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:13.145565 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:13.146663 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:13.146942 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:13.150804 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:13.150820 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:13.150846 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:13.150918 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:13.154899 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:13.155730 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:13.155736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:13.155741 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:13.155911 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:13.156305 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:13.156310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:13.156312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:13.157039 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:13.157346 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:13.157691 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 11:44:13.158123 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":361,"threads_started":1}
I20260808 11:44:13.166568 20038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a23b34ecd304708a67054698b0df649" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44631 } }
I20260808 11:44:13.167001 20038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:13.167078 20038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a23b34ecd304708a67054698b0df649, State: Initialized, Role: FOLLOWER
I20260808 11:44:13.167624 20038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a23b34ecd304708a67054698b0df649" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44631 } }
I20260808 11:44:13.167850 20038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:13.167982 20038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:13.168119 20038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:13.170558 20038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a23b34ecd304708a67054698b0df649" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44631 } }
I20260808 11:44:13.171123 20038 leader_election.cc:302] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a23b34ecd304708a67054698b0df649; no voters:
I20260808 11:44:13.171653 20038 leader_election.cc:288] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:13.171798 20044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:13.172492 20044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [term 1 LEADER]: Becoming Leader. State: Replica: 6a23b34ecd304708a67054698b0df649, State: Running, Role: LEADER
I20260808 11:44:13.173003 20044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a23b34ecd304708a67054698b0df649" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44631 } }
I20260808 11:44:13.173724 20038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:13.176182 20046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a23b34ecd304708a67054698b0df649. Latest consensus state: current_term: 1 leader_uuid: "6a23b34ecd304708a67054698b0df649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a23b34ecd304708a67054698b0df649" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44631 } } }
I20260808 11:44:13.176563 20046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:13.176291 20045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a23b34ecd304708a67054698b0df649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a23b34ecd304708a67054698b0df649" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44631 } } }
I20260808 11:44:13.176713 20045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:13.178364 20054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:13.183612 20054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:13.192483 20054 catalog_manager.cc:1347] Generated new cluster ID: 70bb769b019f4b729c4c09345abffe79
I20260808 11:44:13.192607 20054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:13.204133 20054 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:13.205524 20054 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:13.215372 20054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649: Generated new TSK 0
I20260808 11:44:13.216224 20054 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:44:13.268965 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44631
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:13.605743 20074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:13.606016 20074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:13.606113 20074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:13.613251 20074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:13.613375 20074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:13.613569 20074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:13.623003 20074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:13.624297 20074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:13.625551 20074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:13.638459 20086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:13.638535 20090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:13.638535 20084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:13.639420 20074 server_base.cc:1034] running on GCE node
I20260808 11:44:13.639993 20074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:13.640910 20074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:13.642145 20074 hybrid_clock.cc:648] HybridClock initialized: now 1786189453642099 us; error 48 us; skew 500 ppm
I20260808 11:44:13.642457 20074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:13 dist-test-slave-39l8 krb5kdc[19912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:13.645586 20074 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:44:13.646824 20074 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:13.646951 20074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:13.647284 20074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:13.650053 20074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "4edc90addc19412884be0763a815f04d"
format_stamp: "Formatted at 2026-08-08 11:44:13 on dist-test-slave-39l8"
server_key: "a848e371d26b300f07550c509978ec14"
server_key_iv: "2cf84ec71d4223dbdb219898b2a7d3a1"
server_key_version: "encryptionkey@0"
I20260808 11:44:13.650988 20074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "4edc90addc19412884be0763a815f04d"
format_stamp: "Formatted at 2026-08-08 11:44:13 on dist-test-slave-39l8"
server_key: "a848e371d26b300f07550c509978ec14"
server_key_iv: "2cf84ec71d4223dbdb219898b2a7d3a1"
server_key_version: "encryptionkey@0"
I20260808 11:44:13.655956 20074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 11:44:13.659409 20102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:13.660712 20074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:44:13.660892 20074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "4edc90addc19412884be0763a815f04d"
format_stamp: "Formatted at 2026-08-08 11:44:13 on dist-test-slave-39l8"
server_key: "a848e371d26b300f07550c509978ec14"
server_key_iv: "2cf84ec71d4223dbdb219898b2a7d3a1"
server_key_version: "encryptionkey@0"
I20260808 11:44:13.661123 20074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:13.677656 20074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:13.681618 20074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:13.681891 20074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:13.682879 20074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:13.684139 20074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:13.684271 20074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:13.684391 20074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:13.684477 20074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:13.718681 20074 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:41451
I20260808 11:44:13.718783 20268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:41451 every 8 connection(s)
I20260808 11:44:13.719621 20074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:13.722152 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 20074
I20260808 11:44:13.722384 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:44:13.723186 13945 external_mini_cluster.cc:1428] Setting key 8262c95bf8411a252d7f267ab352c63e
Aug 08 11:44:13 dist-test-slave-39l8 krb5kdc[19912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:13.741381 20274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:13.722068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:50267 (local address 127.13.158.126:44631)
0808 11:44:13.723084 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:13.723091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:13.725582 (+ 2491us) server_negotiation.cc:407] Connection header received
0808 11:44:13.725644 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:13.725647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:13.725713 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:13.725794 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:13.727897 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:13.728540 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:13.730921 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:13.731317 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:13.731415 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:13.735463 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:13.735467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:13.735471 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:13.735502 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:13.737964 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:13.738987 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:13.738991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:13.738994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:13.739107 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:13.739636 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:13.739639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:13.739641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:13.739888 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:13.739997 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:13.740946 (+ 949us) server_negotiation.cc:299] Negotiation successful
0808 11:44:13.741113 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":763,"threads_started":1}
I20260808 11:44:13.742993 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:13.722729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44631 (local address 127.13.158.65:50267)
0808 11:44:13.723598 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:13.723649 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:13.725311 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:13.726055 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:13.726067 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:13.726896 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:13.727697 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:13.727725 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:13.728692 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:13.728700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:13.730772 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:13.730790 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:13.731549 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:13.731558 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:13.731861 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:13.732620 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:13.732655 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:13.735242 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:13.738107 (+ 2865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:13.738119 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:13.738146 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:13.738866 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:13.739223 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:13.739230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:13.739237 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:13.739508 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:13.740026 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:13.740037 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:13.740495 (+ 458us) client_negotiation.cc:769] Sending connection context
0808 11:44:13.740865 (+ 370us) client_negotiation.cc:240] Negotiation successful
0808 11:44:13.741469 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":373,"threads_started":1}
I20260808 11:44:13.744629 20271 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44631
I20260808 11:44:13.744951 20271 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:13.745628 20271 heartbeater.cc:511] Master 127.13.158.126:44631 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
I20260808 11:44:13.747872 19984 ts_manager.cc:194] Registered new tserver with Master: 4edc90addc19412884be0763a815f04d (127.13.158.65:41451)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
I20260808 11:44:13.749526 19984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:50267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:44:13.774032 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44631
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:14.101269 20280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:14.101541 20280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:14.101648 20280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:14.108649 20280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:14.108771 20280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:14.108901 20280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:14.118382 20280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:14.119717 20280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:14.120971 20280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:14.133715 20286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:14.133692 20288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:14.134397 20290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:14.136096 20280 server_base.cc:1034] running on GCE node
I20260808 11:44:14.136656 20280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:14.137622 20280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:14.138897 20280 hybrid_clock.cc:648] HybridClock initialized: now 1786189454138849 us; error 42 us; skew 500 ppm
I20260808 11:44:14.139225 20280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:14 dist-test-slave-39l8 krb5kdc[19912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189454, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:14.146201 20280 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:44:14.147459 20280 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:14.147593 20280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:14.147914 20280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:14.150810 20280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "6af7de8a90a147429e68b73b8c9bf0c4"
format_stamp: "Formatted at 2026-08-08 11:44:14 on dist-test-slave-39l8"
server_key: "c1e37c7e8b26e15cc32c1232241a1e8f"
server_key_iv: "6d78315939a56b38a34e90a3c64407d1"
server_key_version: "encryptionkey@0"
I20260808 11:44:14.151618 20280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "6af7de8a90a147429e68b73b8c9bf0c4"
format_stamp: "Formatted at 2026-08-08 11:44:14 on dist-test-slave-39l8"
server_key: "c1e37c7e8b26e15cc32c1232241a1e8f"
server_key_iv: "6d78315939a56b38a34e90a3c64407d1"
server_key_version: "encryptionkey@0"
I20260808 11:44:14.156582 20280 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:44:14.160032 20298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:14.161322 20280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:44:14.161510 20280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "6af7de8a90a147429e68b73b8c9bf0c4"
format_stamp: "Formatted at 2026-08-08 11:44:14 on dist-test-slave-39l8"
server_key: "c1e37c7e8b26e15cc32c1232241a1e8f"
server_key_iv: "6d78315939a56b38a34e90a3c64407d1"
server_key_version: "encryptionkey@0"
I20260808 11:44:14.161737 20280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:14.197314 20280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:14.202054 20280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:14.202359 20280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:14.203363 20280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:14.204772 20280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:14.204900 20280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:14.205045 20280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:14.205144 20280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:14.238780 20280 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:37001
I20260808 11:44:14.238842 20463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:37001 every 8 connection(s)
I20260808 11:44:14.239703 20280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:14.245055 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 20280
I20260808 11:44:14.245250 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:14.245750 13945 external_mini_cluster.cc:1428] Setting key ebc95654a10ccb76e90638180e3034a5
Aug 08 11:44:14 dist-test-slave-39l8 krb5kdc[19912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189454, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:14.261322 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.242220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:60667 (local address 127.13.158.126:44631)
0808 11:44:14.242646 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:14.242650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:14.243641 (+ 991us) server_negotiation.cc:407] Connection header received
0808 11:44:14.245576 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:14.245583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:14.245675 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:14.245807 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:14.247632 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.248747 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.250258 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.251017 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.251162 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.254383 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:14.254392 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:14.254399 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:14.254444 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:14.257934 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:14.258608 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:14.258615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:14.258622 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:14.258797 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:14.259243 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:14.259250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:14.259256 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:14.259620 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:14.259771 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:14.260536 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 11:44:14.260785 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":221,"threads_started":1}
I20260808 11:44:14.261848 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.242817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44631 (local address 127.13.158.66:60667)
0808 11:44:14.243473 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:14.243508 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:14.245272 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:14.246021 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:14.246029 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:14.246653 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:14.247420 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.247445 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.248898 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.248902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.250108 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.250119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.251216 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.251221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.251394 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.252082 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:14.252103 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:14.254149 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:14.258063 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:14.258070 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:14.258080 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:14.258489 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:14.258939 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:14.258943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:14.258945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:14.259111 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:14.259775 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:14.259783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:14.260157 (+ 374us) client_negotiation.cc:769] Sending connection context
0808 11:44:14.260391 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 11:44:14.260764 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":258,"threads_started":1}
I20260808 11:44:14.263471 20464 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44631
I20260808 11:44:14.263829 20464 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:14.264675 20464 heartbeater.cc:511] Master 127.13.158.126:44631 requested a full tablet report, sending...
I20260808 11:44:14.266455 19984 ts_manager.cc:194] Registered new tserver with Master: 6af7de8a90a147429e68b73b8c9bf0c4 (127.13.158.66:37001)
I20260808 11:44:14.267401 19984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:60667
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:44:14.297564 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44631
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:14.621409 20472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:14.621697 20472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:14.621799 20472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:14.629092 20472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:14.629204 20472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:14.629317 20472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:14.639400 20472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:14.640715 20472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:14.642302 20472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:14.655700 20479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:14.655670 20483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:14.655761 20481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:14.656682 20472 server_base.cc:1034] running on GCE node
I20260808 11:44:14.657348 20472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:14.658339 20472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:14.659588 20472 hybrid_clock.cc:648] HybridClock initialized: now 1786189454659520 us; error 52 us; skew 500 ppm
I20260808 11:44:14.659916 20472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:14 dist-test-slave-39l8 krb5kdc[19912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189454, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:14.663120 20472 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:44:14.664300 20472 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:14.664444 20472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:14.664775 20472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:14.668042 20472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "f60c0ef2ecc04b8590daed1af44b8d09"
format_stamp: "Formatted at 2026-08-08 11:44:14 on dist-test-slave-39l8"
server_key: "ba4fbf58d056e2a162c44e68708bd88c"
server_key_iv: "08646673833e97b5baddd643a05a9ba1"
server_key_version: "encryptionkey@0"
I20260808 11:44:14.668869 20472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "f60c0ef2ecc04b8590daed1af44b8d09"
format_stamp: "Formatted at 2026-08-08 11:44:14 on dist-test-slave-39l8"
server_key: "ba4fbf58d056e2a162c44e68708bd88c"
server_key_iv: "08646673833e97b5baddd643a05a9ba1"
server_key_version: "encryptionkey@0"
I20260808 11:44:14.673974 20472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:44:14.677448 20491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:14.678838 20472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:14.679018 20472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "f60c0ef2ecc04b8590daed1af44b8d09"
format_stamp: "Formatted at 2026-08-08 11:44:14 on dist-test-slave-39l8"
server_key: "ba4fbf58d056e2a162c44e68708bd88c"
server_key_iv: "08646673833e97b5baddd643a05a9ba1"
server_key_version: "encryptionkey@0"
I20260808 11:44:14.679236 20472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:14.702806 20472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:14.706619 20472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:14.706920 20472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:14.707891 20472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:14.709227 20472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:14.709357 20472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:14.709487 20472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:14.709584 20472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:14.744465 20472 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:36421
I20260808 11:44:14.744529 20660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:36421 every 8 connection(s)
I20260808 11:44:14.745321 20472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:14.746466 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 20472
I20260808 11:44:14.746673 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:14.747252 13945 external_mini_cluster.cc:1428] Setting key 90659572fa7cc88b48ee64425aa1f2a6
I20260808 11:44:14.752368 20271 heartbeater.cc:503] Master 127.13.158.126:44631 was elected leader, sending a full tablet report...
Aug 08 11:44:14 dist-test-slave-39l8 krb5kdc[19912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189454, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:14.765367 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.748074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:56671 (local address 127.13.158.126:44631)
0808 11:44:14.748262 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:14.748268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:14.749529 (+ 1261us) server_negotiation.cc:407] Connection header received
0808 11:44:14.751517 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:14.751521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:14.751587 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:14.751684 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:14.753477 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.754219 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.755753 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.756135 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.756271 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.759688 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:14.759692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:14.759696 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:14.759731 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:14.762312 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:14.763143 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:14.763147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:14.763149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:14.763260 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:14.763689 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:14.763696 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:14.763704 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:14.763995 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:14.764153 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:14.764821 (+ 668us) server_negotiation.cc:299] Negotiation successful
0808 11:44:14.765033 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:44:14.766276 20667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.748517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44631 (local address 127.13.158.67:56671)
0808 11:44:14.749322 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:14.749391 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:14.751281 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:14.751877 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:14.751884 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:14.752614 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:14.753284 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.753301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.754377 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.754381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.755608 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.755619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.756355 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.756365 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.756586 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.757193 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:14.757222 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:14.759485 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:14.762449 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:14.762457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:14.762470 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:14.763015 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:14.763367 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:14.763372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:14.763375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:14.763568 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:14.764128 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:14.764135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:14.764448 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 11:44:14.764692 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 11:44:14.765247 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":370,"threads_started":1}
I20260808 11:44:14.767598 20662 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44631
I20260808 11:44:14.767849 20662 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:14.768401 20662 heartbeater.cc:511] Master 127.13.158.126:44631 requested a full tablet report, sending...
I20260808 11:44:14.769688 19984 ts_manager.cc:194] Registered new tserver with Master: f60c0ef2ecc04b8590daed1af44b8d09 (127.13.158.67:36421)
I20260808 11:44:14.770486 19984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:56671
I20260808 11:44:14.775053 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:44:14.792627 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.779624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.13.158.126:44631)
0808 11:44:14.779801 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:14.779806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:14.780386 (+ 580us) server_negotiation.cc:407] Connection header received
0808 11:44:14.780602 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:14.780608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:14.780678 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:14.780780 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:14.782217 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.783082 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.784379 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.784806 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.784942 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.787322 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:14.787327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:14.787330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:14.787361 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:14.789894 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:14.790577 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:14.790584 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:14.790588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:14.790774 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:14.791156 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:14.791164 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:14.791167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:14.791473 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:14.791733 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:14.792081 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 11:44:14.792284 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:44:14.796563 19984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:44:14.803886 19984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:14.827255 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.817389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52788 (local address 127.13.158.66:37001)
0808 11:44:14.818040 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:14.818052 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:14.818667 (+ 615us) server_negotiation.cc:407] Connection header received
0808 11:44:14.818881 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:14.818893 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:14.819150 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:14.819363 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:14.820309 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.821660 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.824539 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.825996 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.826121 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.826340 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:14.826769 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 11:44:14.826921 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":616,"threads_started":1}
I20260808 11:44:14.827306 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.816181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:37001 (local address 127.0.0.1:52788)
0808 11:44:14.818481 (+ 2300us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:14.818506 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:14.818677 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:14.819621 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:14.819628 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:14.819655 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:14.820106 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.820121 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.821817 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.821822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.824340 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.824358 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.826302 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.826308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.826482 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.826516 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:44:14.826679 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 11:44:14.826885 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2139,"mutex_wait_us":1,"thread_start_us":342,"threads_started":1}
I20260808 11:44:14.828090 20683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.816127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39858 (local address 127.13.158.67:36421)
0808 11:44:14.819668 (+ 3541us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:14.819677 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:14.819711 (+ 34us) server_negotiation.cc:407] Connection header received
0808 11:44:14.819804 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:14.819810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:14.820029 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:14.820601 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:14.821413 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.822554 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.825240 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.826151 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.826304 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.826515 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:14.827392 (+ 877us) server_negotiation.cc:299] Negotiation successful
0808 11:44:14.827553 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3366,"thread_start_us":3230,"threads_started":1}
I20260808 11:44:14.827905 20681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.815778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:36421 (local address 127.0.0.1:39858)
0808 11:44:14.816970 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:14.817009 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:14.817183 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:14.820260 (+ 3077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:14.820265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:14.820286 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:14.821172 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.821193 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.822689 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.822694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.825082 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.825094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.826728 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.826735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.826907 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.826942 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 11:44:14.827035 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 11:44:14.827157 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":413,"thread_start_us":2564,"threads_started":1}
I20260808 11:44:14.829105 20684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.816680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:41451 (local address 127.0.0.1:50810)
0808 11:44:14.819307 (+ 2627us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:14.819328 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:14.819457 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:14.820362 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:14.820368 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:14.820403 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:14.821293 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.821308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.823237 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.823242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.826031 (+ 2789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:14.826046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.828287 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.828302 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:14.828483 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.828506 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:44:14.828612 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 11:44:14.828719 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2464,"mutex_wait_us":165,"thread_start_us":2272,"threads_started":1}
I20260808 11:44:14.829171 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:14.817844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.13.158.65:41451)
0808 11:44:14.818415 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:14.818424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:14.819570 (+ 1146us) server_negotiation.cc:407] Connection header received
0808 11:44:14.819665 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:14.819672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:14.819880 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:14.820082 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:14.821482 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.823062 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.826218 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:14.827655 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:14.827803 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:14.828048 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:14.828671 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 11:44:14.828848 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":372,"threads_started":1}
I20260808 11:44:14.832556 20369 tablet_service.cc:1467] Processing CreateTablet for tablet 57f0c4537b8a4dccab24d022eaf48b07 (DEFAULT_TABLE table=test-table [id=b4e1d92fd75d48b1bcbe4c23dfef776d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:14.832463 20185 tablet_service.cc:1467] Processing CreateTablet for tablet 57f0c4537b8a4dccab24d022eaf48b07 (DEFAULT_TABLE table=test-table [id=b4e1d92fd75d48b1bcbe4c23dfef776d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:14.832556 20559 tablet_service.cc:1467] Processing CreateTablet for tablet 57f0c4537b8a4dccab24d022eaf48b07 (DEFAULT_TABLE table=test-table [id=b4e1d92fd75d48b1bcbe4c23dfef776d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:14.833911 20185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f0c4537b8a4dccab24d022eaf48b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:14.833945 20559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f0c4537b8a4dccab24d022eaf48b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:14.833945 20369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f0c4537b8a4dccab24d022eaf48b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:14.844630 20691 tablet_bootstrap.cc:492] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Bootstrap starting.
I20260808 11:44:14.848297 20692 tablet_bootstrap.cc:492] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Bootstrap starting.
I20260808 11:44:14.849131 20691 tablet_bootstrap.cc:654] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:14.850545 20691 log.cc:824] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:14.851962 20692 tablet_bootstrap.cc:654] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:14.852098 20695 tablet_bootstrap.cc:492] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Bootstrap starting.
I20260808 11:44:14.853276 20692 log.cc:824] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:14.854009 20691 tablet_bootstrap.cc:492] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: No bootstrap required, opened a new log
I20260808 11:44:14.854305 20691 ts_tablet_manager.cc:1397] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 11:44:14.856526 20692 tablet_bootstrap.cc:492] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: No bootstrap required, opened a new log
I20260808 11:44:14.856760 20695 tablet_bootstrap.cc:654] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:14.856829 20692 ts_tablet_manager.cc:1397] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260808 11:44:14.858072 20695 log.cc:824] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:14.861358 20695 tablet_bootstrap.cc:492] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: No bootstrap required, opened a new log
I20260808 11:44:14.861608 20695 ts_tablet_manager.cc:1397] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 11:44:14.862401 20691 raft_consensus.cc:348] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:14.862792 20691 raft_consensus.cc:374] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:14.862923 20691 raft_consensus.cc:729] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4edc90addc19412884be0763a815f04d, State: Initialized, Role: FOLLOWER
I20260808 11:44:14.863541 20691 consensus_queue.cc:260] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:14.865319 20691 ts_tablet_manager.cc:1428] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Time spent starting tablet: real 0.011s user 0.009s sys 0.004s
I20260808 11:44:14.867142 20692 raft_consensus.cc:348] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:14.867584 20692 raft_consensus.cc:374] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:14.867702 20692 raft_consensus.cc:729] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af7de8a90a147429e68b73b8c9bf0c4, State: Initialized, Role: FOLLOWER
I20260808 11:44:14.868306 20692 consensus_queue.cc:260] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:14.869964 20464 heartbeater.cc:503] Master 127.13.158.126:44631 was elected leader, sending a full tablet report...
I20260808 11:44:14.870553 20692 ts_tablet_manager.cc:1428] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260808 11:44:14.874168 20695 raft_consensus.cc:348] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:14.874641 20695 raft_consensus.cc:374] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:14.874778 20695 raft_consensus.cc:729] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f60c0ef2ecc04b8590daed1af44b8d09, State: Initialized, Role: FOLLOWER
I20260808 11:44:14.875298 20695 consensus_queue.cc:260] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:14.876315 20662 heartbeater.cc:503] Master 127.13.158.126:44631 was elected leader, sending a full tablet report...
I20260808 11:44:14.876689 20695 ts_tablet_manager.cc:1428] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
W20260808 11:44:14.971825 20273 tablet.cc:2369] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:44:14.991703 20465 tablet.cc:2369] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:44:14.996914 20664 tablet.cc:2369] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:44:15.087598 20699 raft_consensus.cc:482] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:15.087917 20699 raft_consensus.cc:504] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:15.089625 20699 leader_election.cc:288] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f60c0ef2ecc04b8590daed1af44b8d09 (127.13.158.67:36421), 6af7de8a90a147429e68b73b8c9bf0c4 (127.13.158.66:37001)
I20260808 11:44:15.096752 20683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:15.090087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:54121 (local address 127.13.158.67:36421)
0808 11:44:15.090338 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:15.090347 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:15.091215 (+ 868us) server_negotiation.cc:407] Connection header received
0808 11:44:15.091467 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:15.091471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:15.091531 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:15.091623 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:15.092339 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.093032 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.095016 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.095795 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.095977 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:15.096050 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:15.096370 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 11:44:15.096494 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:44:15.096869 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:15.090090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:48969 (local address 127.13.158.66:37001)
0808 11:44:15.090337 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:15.090346 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:15.091184 (+ 838us) server_negotiation.cc:407] Connection header received
0808 11:44:15.091408 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:15.091412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:15.091472 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:15.091570 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:15.092187 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.092903 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.094922 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.095900 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.096076 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:15.096150 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:15.096479 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 11:44:15.096600 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 11:44:15.096906 20705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:15.089976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:36421 (local address 127.13.158.65:54121)
0808 11:44:15.091095 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:15.091112 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:15.091318 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:15.091659 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:15.091669 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:15.091726 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:15.092178 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:15.092188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.093179 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.093186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:15.094854 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:15.094866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.096001 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.096007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:15.096175 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:15.096203 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 11:44:15.096342 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 11:44:15.096465 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"spinlock_wait_cycles":52352,"thread_start_us":750,"threads_started":1}
I20260808 11:44:15.097324 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:15.089911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:37001 (local address 127.13.158.65:48969)
0808 11:44:15.091045 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:15.091075 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:15.091259 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:15.091585 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:15.091589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:15.091611 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:15.091995 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:15.092015 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.093029 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.093037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:15.094755 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:15.094775 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.096113 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.096123 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:15.096301 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:15.096323 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:44:15.096453 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 11:44:15.096575 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":41,"thread_start_us":705,"threads_started":1}
I20260808 11:44:15.098007 20591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f0c4537b8a4dccab24d022eaf48b07" candidate_uuid: "4edc90addc19412884be0763a815f04d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" is_pre_election: true
I20260808 11:44:15.098304 20396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f0c4537b8a4dccab24d022eaf48b07" candidate_uuid: "4edc90addc19412884be0763a815f04d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" is_pre_election: true
I20260808 11:44:15.098510 20591 raft_consensus.cc:2454] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4edc90addc19412884be0763a815f04d in term 0.
I20260808 11:44:15.098853 20396 raft_consensus.cc:2454] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4edc90addc19412884be0763a815f04d in term 0.
I20260808 11:44:15.099263 20103 leader_election.cc:302] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4edc90addc19412884be0763a815f04d, f60c0ef2ecc04b8590daed1af44b8d09; no voters:
I20260808 11:44:15.099707 20699 raft_consensus.cc:2781] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:15.099824 20699 raft_consensus.cc:482] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:15.099897 20699 raft_consensus.cc:3037] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:15.102140 20699 raft_consensus.cc:504] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:15.102748 20699 leader_election.cc:288] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [CANDIDATE]: Term 1 election: Requested vote from peers f60c0ef2ecc04b8590daed1af44b8d09 (127.13.158.67:36421), 6af7de8a90a147429e68b73b8c9bf0c4 (127.13.158.66:37001)
I20260808 11:44:15.103075 20591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f0c4537b8a4dccab24d022eaf48b07" candidate_uuid: "4edc90addc19412884be0763a815f04d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f60c0ef2ecc04b8590daed1af44b8d09"
I20260808 11:44:15.103272 20591 raft_consensus.cc:3037] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:15.103271 20396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f0c4537b8a4dccab24d022eaf48b07" candidate_uuid: "4edc90addc19412884be0763a815f04d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6af7de8a90a147429e68b73b8c9bf0c4"
I20260808 11:44:15.103470 20396 raft_consensus.cc:3037] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:15.105471 20591 raft_consensus.cc:2454] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4edc90addc19412884be0763a815f04d in term 1.
I20260808 11:44:15.105829 20103 leader_election.cc:302] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4edc90addc19412884be0763a815f04d, f60c0ef2ecc04b8590daed1af44b8d09; no voters:
I20260808 11:44:15.105976 20396 raft_consensus.cc:2454] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4edc90addc19412884be0763a815f04d in term 1.
I20260808 11:44:15.106094 20699 raft_consensus.cc:2781] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:15.106642 20699 raft_consensus.cc:686] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 1 LEADER]: Becoming Leader. State: Replica: 4edc90addc19412884be0763a815f04d, State: Running, Role: LEADER
I20260808 11:44:15.107147 20699 consensus_queue.cc:237] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } }
I20260808 11:44:15.111696 19983 catalog_manager.cc:5495] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4edc90addc19412884be0763a815f04d (127.13.158.65). New cstate: current_term: 1 leader_uuid: "4edc90addc19412884be0763a815f04d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4edc90addc19412884be0763a815f04d" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41451 } health_report { overall_health: HEALTHY } } }
I20260808 11:44:15.194873 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:15.189077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.13.158.65:41451)
0808 11:44:15.189314 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:15.189321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:15.189431 (+ 110us) server_negotiation.cc:407] Connection header received
0808 11:44:15.189696 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:15.189700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:15.189790 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:15.190051 (+ 261us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:15.190455 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.191293 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.192669 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.193090 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.193230 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:15.193726 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:15.194248 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:15.194413 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:15.194494 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 11:44:15.194585 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:44:15.203069 20396 raft_consensus.cc:1261] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 1 FOLLOWER]: Refusing update from remote peer 4edc90addc19412884be0763a815f04d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:15.203063 20591 raft_consensus.cc:1261] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 1 FOLLOWER]: Refusing update from remote peer 4edc90addc19412884be0763a815f04d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:15.204078 20706 consensus_queue.cc:1035] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6af7de8a90a147429e68b73b8c9bf0c4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:15.204440 20699 consensus_queue.cc:1035] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f60c0ef2ecc04b8590daed1af44b8d09" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:15.229750 19983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:15.230038 19983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:15.233618 19983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a23b34ecd304708a67054698b0df649: Sending DeleteTablet for 3 replicas of tablet 57f0c4537b8a4dccab24d022eaf48b07
I20260808 11:44:15.234905 20369 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f0c4537b8a4dccab24d022eaf48b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:15 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:52788
I20260808 11:44:15.234853 20559 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f0c4537b8a4dccab24d022eaf48b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:15 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:39858
I20260808 11:44:15.234983 20185 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f0c4537b8a4dccab24d022eaf48b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:15 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:50810
I20260808 11:44:15.237666 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 20074
I20260808 11:44:15.243240 20723 tablet_replica.cc:330] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: stopping tablet replica
I20260808 11:44:15.243669 20723 raft_consensus.cc:2229] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:15.244207 20723 raft_consensus.cc:2258] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:15.244861 20726 tablet_replica.cc:330] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: stopping tablet replica
I20260808 11:44:15.245206 20726 raft_consensus.cc:2229] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:44:15.245764 20726 raft_consensus.cc:2258] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:15.248588 20726 ts_tablet_manager.cc:1905] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:15.254256 20723 ts_tablet_manager.cc:1905] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:15.263293 20724 tablet_replica.cc:330] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: stopping tablet replica
I20260808 11:44:15.263796 20724 raft_consensus.cc:2229] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:15.264335 20724 raft_consensus.cc:2258] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:15.264832 20726 ts_tablet_manager.cc:1918] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:15.265027 20726 log.cc:1196] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/57f0c4537b8a4dccab24d022eaf48b07
I20260808 11:44:15.265695 20726 ts_tablet_manager.cc:1939] T 57f0c4537b8a4dccab24d022eaf48b07 P 4edc90addc19412884be0763a815f04d: Deleting consensus metadata
I20260808 11:44:15.268105 19956 catalog_manager.cc:4841] TS 4edc90addc19412884be0763a815f04d (127.13.158.65:41451): tablet 57f0c4537b8a4dccab24d022eaf48b07 (table test-table [id=b4e1d92fd75d48b1bcbe4c23dfef776d]) successfully deleted
I20260808 11:44:15.268630 20724 ts_tablet_manager.cc:1905] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:15.268815 20723 ts_tablet_manager.cc:1918] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:15.269032 20723 log.cc:1196] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/57f0c4537b8a4dccab24d022eaf48b07
I20260808 11:44:15.269650 20723 ts_tablet_manager.cc:1939] T 57f0c4537b8a4dccab24d022eaf48b07 P 6af7de8a90a147429e68b73b8c9bf0c4: Deleting consensus metadata
I20260808 11:44:15.271597 19955 catalog_manager.cc:4841] TS 6af7de8a90a147429e68b73b8c9bf0c4 (127.13.158.66:37001): tablet 57f0c4537b8a4dccab24d022eaf48b07 (table test-table [id=b4e1d92fd75d48b1bcbe4c23dfef776d]) successfully deleted
Aug 08 11:44:15 dist-test-slave-39l8 krb5kdc[19912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:44:15.275760 20724 ts_tablet_manager.cc:1918] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:15.275930 20724 log.cc:1196] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/57f0c4537b8a4dccab24d022eaf48b07
I20260808 11:44:15.276449 20724 ts_tablet_manager.cc:1939] T 57f0c4537b8a4dccab24d022eaf48b07 P f60c0ef2ecc04b8590daed1af44b8d09: Deleting consensus metadata
I20260808 11:44:15.278229 19954 catalog_manager.cc:4841] TS f60c0ef2ecc04b8590daed1af44b8d09 (127.13.158.67:36421): tablet 57f0c4537b8a4dccab24d022eaf48b07 (table test-table [id=b4e1d92fd75d48b1bcbe4c23dfef776d]) successfully deleted
I20260808 11:44:15.279958 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:15.239537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.13.158.65:41451)
0808 11:44:15.239766 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:15.239775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:15.255559 (+ 15784us) server_negotiation.cc:407] Connection header received
0808 11:44:15.256179 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:15.256188 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:15.256300 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:15.256403 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:15.261662 (+ 5259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.263205 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.266165 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.266934 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.267040 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:15.273477 (+ 6437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:15.273493 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:15.273506 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:15.273548 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:15.276927 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:15.277645 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:15.277656 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:15.277658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:15.277808 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:15.278248 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:15.278252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:15.278254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:15.278670 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:15.279029 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:15.279342 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 11:44:15.279585 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:44:15.280776 20263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:15.378746 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 20074
I20260808 11:44:15.406804 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 20280
Aug 08 11:44:15 dist-test-slave-39l8 krb5kdc[19912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:44:15.421283 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:15.407546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.13.158.66:37001)
0808 11:44:15.407743 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:15.407748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:15.407766 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:44:15.407912 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:15.407916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:15.407978 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:15.408087 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:15.409263 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.409951 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.411199 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.411598 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.411692 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:15.415705 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:15.415718 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:15.415734 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:15.415768 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:15.418363 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:15.419161 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:15.419168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:15.419170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:15.419302 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:15.419685 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:15.419689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:15.419691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:15.420075 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:15.420403 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:15.420720 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 11:44:15.420944 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:44:15.421962 20458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:15.509790 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 20280
I20260808 11:44:15.534811 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 20472
Aug 08 11:44:15 dist-test-slave-39l8 krb5kdc[19912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:44:15.549561 20683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:15.535593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.13.158.67:36421)
0808 11:44:15.535837 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:15.535843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:15.535863 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:44:15.535985 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:15.535989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:15.536053 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:15.536147 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:15.537398 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.538104 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.539437 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:15.539862 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:15.539977 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:15.544386 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:15.544402 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:15.544418 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:15.544448 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:15.547178 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:15.547783 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:15.547790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:15.547792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:15.547916 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:15.548240 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:15.548243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:15.548245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:15.548573 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:15.548817 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:15.549048 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 11:44:15.549200 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260808 11:44:15.550335 20655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:15.638962 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 20472
I20260808 11:44:15.663031 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 19930
I20260808 11:44:15.664242 20031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:15.775038 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 19930
2026-08-08T11:44:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4530 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T11:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:15Z Disabled control of system clock
I20260808 11:44:15.851939 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44795
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:44795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 11:44:16.181167 20739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:16.181443 20739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:16.181562 20739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:16.188830 20739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:16.188932 20739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:16.188970 20739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:16.189025 20739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:16.189072 20739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:16.197984 20739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:44795
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:16.199393 20739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:16.201089 20739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:16.213075 20745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:16.213111 20749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:16.213369 20746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:16.214033 20739 server_base.cc:1034] running on GCE node
I20260808 11:44:16.214890 20739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:16.216264 20739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:16.217500 20739 hybrid_clock.cc:648] HybridClock initialized: now 1786189456217467 us; error 63 us; skew 500 ppm
I20260808 11:44:16.217847 20739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:44:16.220347 20739 webserver.cc:460] Webserver started at http://127.13.158.126:33447/ using document root <none> and password file <none>
I20260808 11:44:16.221055 20739 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:16.221192 20739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:16.221541 20739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:16.224558 20739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "c80633a1dade49bf874baa789c44817b"
format_stamp: "Formatted at 2026-08-08 11:44:16 on dist-test-slave-39l8"
server_key: "77894f08b9bfd06c04c77f5f2b05c85b"
server_key_iv: "4912702581eba65b66ce48bef84d9182"
server_key_version: "encryptionkey@0"
I20260808 11:44:16.225350 20739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "c80633a1dade49bf874baa789c44817b"
format_stamp: "Formatted at 2026-08-08 11:44:16 on dist-test-slave-39l8"
server_key: "77894f08b9bfd06c04c77f5f2b05c85b"
server_key_iv: "4912702581eba65b66ce48bef84d9182"
server_key_version: "encryptionkey@0"
I20260808 11:44:16.230387 20739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:44:16.233793 20758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:16.235112 20739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:16.235298 20739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c80633a1dade49bf874baa789c44817b"
format_stamp: "Formatted at 2026-08-08 11:44:16 on dist-test-slave-39l8"
server_key: "77894f08b9bfd06c04c77f5f2b05c85b"
server_key_iv: "4912702581eba65b66ce48bef84d9182"
server_key_version: "encryptionkey@0"
I20260808 11:44:16.235526 20739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:16.241164 20739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:16.241918 20739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:16.242229 20739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:16.260833 20739 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:44795
I20260808 11:44:16.260869 20829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:44795 every 8 connection(s)
I20260808 11:44:16.262282 20739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:16.266075 20831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:16.269775 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 20739
I20260808 11:44:16.269971 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:16.270612 13945 external_mini_cluster.cc:1428] Setting key 5da365229395fa462eed5575012fe271
I20260808 11:44:16.276823 20831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b: Bootstrap starting.
I20260808 11:44:16.279779 20831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:16.280843 20834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:16.272597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54408 (local address 127.13.158.126:44795)
0808 11:44:16.273569 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:16.273583 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:16.273623 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:44:16.275509 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:16.275530 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:16.275904 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:16.276260 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:16.279241 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:16.279247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:16.279259 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:16.279276 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:16.279326 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:16.279340 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:16.279554 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:16.279718 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 11:44:16.279989 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":356,"threads_started":1}
I20260808 11:44:16.280891 20831 log.cc:824] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:16.284072 20831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b: No bootstrap required, opened a new log
I20260808 11:44:16.291373 20831 raft_consensus.cc:348] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80633a1dade49bf874baa789c44817b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44795 } }
I20260808 11:44:16.291672 20831 raft_consensus.cc:374] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:16.291744 20831 raft_consensus.cc:729] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80633a1dade49bf874baa789c44817b, State: Initialized, Role: FOLLOWER
I20260808 11:44:16.292322 20831 consensus_queue.cc:260] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80633a1dade49bf874baa789c44817b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44795 } }
I20260808 11:44:16.292520 20831 raft_consensus.cc:388] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:16.292616 20831 raft_consensus.cc:482] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:16.292743 20831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:16.294845 20831 raft_consensus.cc:504] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80633a1dade49bf874baa789c44817b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44795 } }
I20260808 11:44:16.295279 20831 leader_election.cc:302] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c80633a1dade49bf874baa789c44817b; no voters:
I20260808 11:44:16.295770 20831 leader_election.cc:288] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:16.295945 20837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:16.296860 20837 raft_consensus.cc:686] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [term 1 LEADER]: Becoming Leader. State: Replica: c80633a1dade49bf874baa789c44817b, State: Running, Role: LEADER
I20260808 11:44:16.297413 20837 consensus_queue.cc:237] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80633a1dade49bf874baa789c44817b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44795 } }
I20260808 11:44:16.298228 20831 sys_catalog.cc:564] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:16.300879 20841 sys_catalog.cc:455] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c80633a1dade49bf874baa789c44817b. Latest consensus state: current_term: 1 leader_uuid: "c80633a1dade49bf874baa789c44817b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80633a1dade49bf874baa789c44817b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44795 } } }
I20260808 11:44:16.300920 20840 sys_catalog.cc:455] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c80633a1dade49bf874baa789c44817b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80633a1dade49bf874baa789c44817b" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44795 } } }
I20260808 11:44:16.301278 20841 sys_catalog.cc:458] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:16.301359 20840 sys_catalog.cc:458] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:16.303551 20848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:16.308931 20848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:16.317642 20848 catalog_manager.cc:1347] Generated new cluster ID: 247ce5dfb33b491592f264bf2d2d51a3
I20260808 11:44:16.317752 20848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:16.341221 20848 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:16.341671 20848 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:16.348974 20848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b: Generated new TSK 0
I20260808 11:44:16.349977 20848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:44:16.362571 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44795
--builtin_ntp_servers=127.13.158.84:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 11:44:16.695220 20861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:16.695518 20861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:16.695665 20861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:16.703481 20861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:16.703612 20861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:16.703824 20861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:16.712987 20861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:16.714279 20861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:16.716004 20861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:16.730034 20871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:16.730072 20869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:16.730123 20873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:16.731225 20861 server_base.cc:1034] running on GCE node
I20260808 11:44:16.731886 20861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:16.732831 20861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:16.734035 20861 hybrid_clock.cc:648] HybridClock initialized: now 1786189456733973 us; error 58 us; skew 500 ppm
I20260808 11:44:16.734411 20861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:44:16.736691 20861 webserver.cc:460] Webserver started at http://127.13.158.65:35605/ using document root <none> and password file <none>
I20260808 11:44:16.737407 20861 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:16.737527 20861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:16.737856 20861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:16.741078 20861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "b278b3b19cab45e2b0140f631c15e980"
format_stamp: "Formatted at 2026-08-08 11:44:16 on dist-test-slave-39l8"
server_key: "102e5506eef57cd5778493c319e17650"
server_key_iv: "4d3152149cb85731ca507b57a869dc5e"
server_key_version: "encryptionkey@0"
I20260808 11:44:16.741923 20861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "b278b3b19cab45e2b0140f631c15e980"
format_stamp: "Formatted at 2026-08-08 11:44:16 on dist-test-slave-39l8"
server_key: "102e5506eef57cd5778493c319e17650"
server_key_iv: "4d3152149cb85731ca507b57a869dc5e"
server_key_version: "encryptionkey@0"
I20260808 11:44:16.746994 20861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 11:44:16.750278 20882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:16.751595 20861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:16.751778 20861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "b278b3b19cab45e2b0140f631c15e980"
format_stamp: "Formatted at 2026-08-08 11:44:16 on dist-test-slave-39l8"
server_key: "102e5506eef57cd5778493c319e17650"
server_key_iv: "4d3152149cb85731ca507b57a869dc5e"
server_key_version: "encryptionkey@0"
I20260808 11:44:16.752017 20861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:16.757591 20861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:16.758354 20861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:16.758646 20861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:16.759627 20861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:16.761027 20861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:16.761160 20861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:16.761302 20861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:16.761395 20861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:16.795410 20861 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:33981
I20260808 11:44:16.795465 21039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:33981 every 8 connection(s)
I20260808 11:44:16.796779 20861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:16.804777 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 20861
I20260808 11:44:16.804979 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:44:16.805135 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:16.799386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:36175 (local address 127.13.158.126:44795)
0808 11:44:16.799859 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:16.799863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:16.801061 (+ 1198us) server_negotiation.cc:407] Connection header received
0808 11:44:16.802800 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:16.802805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:16.802880 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:16.802996 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:16.804335 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:16.804340 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:16.804345 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:16.804363 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:16.804412 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:16.804423 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:16.804556 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:16.804763 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 11:44:16.804833 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":231,"threads_started":1}
I20260808 11:44:16.805768 13945 external_mini_cluster.cc:1428] Setting key 3a047f2cc4df56ff5daeb9e933cb5c7a
I20260808 11:44:16.806067 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:16.800030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44795 (local address 127.13.158.65:36175)
0808 11:44:16.800899 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:16.800934 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:16.802514 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:16.803261 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:16.803269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:16.803353 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:16.804052 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:44:16.804054 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:16.804090 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:44:16.804167 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:16.804525 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:16.804538 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:16.804545 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 11:44:16.804841 (+ 296us) client_negotiation.cc:240] Negotiation successful
0808 11:44:16.805071 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":260,"threads_started":1}
I20260808 11:44:16.807675 21041 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44795
I20260808 11:44:16.807993 21041 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:16.808736 21041 heartbeater.cc:511] Master 127.13.158.126:44795 requested a full tablet report, sending...
I20260808 11:44:16.809057 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44795
--builtin_ntp_servers=127.13.158.84:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 11:44:16.811142 20785 ts_manager.cc:194] Registered new tserver with Master: b278b3b19cab45e2b0140f631c15e980 (127.13.158.65:33981)
W20260808 11:44:17.135970 21046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:17.136271 21046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:17.136360 21046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:17.143684 21046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:17.143787 21046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:17.143891 21046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:17.153056 21046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:17.154374 21046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:17.156203 21046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:17.169360 21051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:17.169454 21053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:17.169394 21055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:17.170542 21046 server_base.cc:1034] running on GCE node
I20260808 11:44:17.171192 21046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:17.172135 21046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:17.173367 21046 hybrid_clock.cc:648] HybridClock initialized: now 1786189457173329 us; error 48 us; skew 500 ppm
I20260808 11:44:17.173686 21046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:44:17.180195 21046 webserver.cc:460] Webserver started at http://127.13.158.66:37571/ using document root <none> and password file <none>
I20260808 11:44:17.180962 21046 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:17.181102 21046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:17.181449 21046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:17.184376 21046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "46b6721715c94200afa128fdda2c22bb"
format_stamp: "Formatted at 2026-08-08 11:44:17 on dist-test-slave-39l8"
server_key: "105d74a7beb23ab2f43c5e25e312e1ad"
server_key_iv: "c247ae61f3ab2166318d04e473e9a8fc"
server_key_version: "encryptionkey@0"
I20260808 11:44:17.185175 21046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "46b6721715c94200afa128fdda2c22bb"
format_stamp: "Formatted at 2026-08-08 11:44:17 on dist-test-slave-39l8"
server_key: "105d74a7beb23ab2f43c5e25e312e1ad"
server_key_iv: "c247ae61f3ab2166318d04e473e9a8fc"
server_key_version: "encryptionkey@0"
I20260808 11:44:17.190269 21046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:44:17.193730 21064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:17.195079 21046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 11:44:17.195254 21046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "46b6721715c94200afa128fdda2c22bb"
format_stamp: "Formatted at 2026-08-08 11:44:17 on dist-test-slave-39l8"
server_key: "105d74a7beb23ab2f43c5e25e312e1ad"
server_key_iv: "c247ae61f3ab2166318d04e473e9a8fc"
server_key_version: "encryptionkey@0"
I20260808 11:44:17.195473 21046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:17.201258 21046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:17.202010 21046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:17.202278 21046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:17.203271 21046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:17.204851 21046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:17.204973 21046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:17.205106 21046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:17.205193 21046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:17.237265 21046 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:33783
I20260808 11:44:17.237316 21247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:33783 every 8 connection(s)
I20260808 11:44:17.238626 21046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:17.241750 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 21046
I20260808 11:44:17.241927 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:17.242533 13945 external_mini_cluster.cc:1428] Setting key 3a775e8d94981098de16740fc938cb87
I20260808 11:44:17.245944 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44795
--builtin_ntp_servers=127.13.158.84:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 11:44:17.247728 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.241415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:39375 (local address 127.13.158.126:44795)
0808 11:44:17.241639 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.241645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.243295 (+ 1650us) server_negotiation.cc:407] Connection header received
0808 11:44:17.245415 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.245419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.245478 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.245556 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.247070 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.247074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.247077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.247092 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.247123 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.247134 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.247236 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.247463 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.247510 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 11:44:17.249009 21253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.241953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44795 (local address 127.13.158.66:39375)
0808 11:44:17.243097 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:17.243158 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:17.245127 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:17.245832 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:17.245843 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:17.245967 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:17.246695 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:44:17.246722 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:17.246778 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:44:17.246898 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:17.247236 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:17.247249 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:17.247261 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 11:44:17.247555 (+ 294us) client_negotiation.cc:240] Negotiation successful
0808 11:44:17.247803 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":297,"threads_started":1}
I20260808 11:44:17.250392 21248 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44795
I20260808 11:44:17.250684 21248 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:17.251298 21248 heartbeater.cc:511] Master 127.13.158.126:44795 requested a full tablet report, sending...
I20260808 11:44:17.252640 20785 ts_manager.cc:194] Registered new tserver with Master: 46b6721715c94200afa128fdda2c22bb (127.13.158.66:33783)
W20260808 11:44:17.590910 21256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:17.591174 21256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:17.591321 21256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:17.598533 21256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:17.598685 21256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:17.598884 21256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:17.607820 21256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:17.609187 21256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:17.611063 21256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:17.624141 21262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:17.624071 21263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:17.624179 21267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:17.625039 21256 server_base.cc:1034] running on GCE node
I20260808 11:44:17.625689 21256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:17.626634 21256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:17.627878 21256 hybrid_clock.cc:648] HybridClock initialized: now 1786189457627838 us; error 35 us; skew 500 ppm
I20260808 11:44:17.628192 21256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:44:17.630498 21256 webserver.cc:460] Webserver started at http://127.13.158.67:37853/ using document root <none> and password file <none>
I20260808 11:44:17.631268 21256 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:17.631408 21256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:17.631757 21256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:17.634536 21256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "54fea3684ac04c4ab605633f84d036e2"
format_stamp: "Formatted at 2026-08-08 11:44:17 on dist-test-slave-39l8"
server_key: "599e3712d6980be7ee5c0d4ec729ff70"
server_key_iv: "9eee7db9b1cb397b62d3472621d21359"
server_key_version: "encryptionkey@0"
I20260808 11:44:17.635422 21256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "54fea3684ac04c4ab605633f84d036e2"
format_stamp: "Formatted at 2026-08-08 11:44:17 on dist-test-slave-39l8"
server_key: "599e3712d6980be7ee5c0d4ec729ff70"
server_key_iv: "9eee7db9b1cb397b62d3472621d21359"
server_key_version: "encryptionkey@0"
I20260808 11:44:17.640568 21256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:44:17.643926 21273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:17.645205 21256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:44:17.645372 21256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "54fea3684ac04c4ab605633f84d036e2"
format_stamp: "Formatted at 2026-08-08 11:44:17 on dist-test-slave-39l8"
server_key: "599e3712d6980be7ee5c0d4ec729ff70"
server_key_iv: "9eee7db9b1cb397b62d3472621d21359"
server_key_version: "encryptionkey@0"
I20260808 11:44:17.645587 21256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:17.651489 21256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:17.652277 21256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:17.652551 21256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:17.653456 21256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:17.654925 21256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:17.655045 21256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:17.655191 21256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:17.655285 21256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:17.688452 21256 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:37019
I20260808 11:44:17.688515 21427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:37019 every 8 connection(s)
I20260808 11:44:17.689838 21256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:17.697607 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.692445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:58589 (local address 127.13.158.126:44795)
0808 11:44:17.692675 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.692682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.694008 (+ 1326us) server_negotiation.cc:407] Connection header received
0808 11:44:17.695729 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.695733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.695783 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.695867 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.697022 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.697026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.697029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.697044 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.697075 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.697086 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.697183 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.697346 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.697397 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 11:44:17.698518 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.692988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44795 (local address 127.13.158.67:58589)
0808 11:44:17.693857 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:17.693897 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:17.695480 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:17.696062 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:17.696069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:17.696154 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:17.696751 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:44:17.696753 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:17.696787 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:44:17.696867 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:17.697177 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:17.697185 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:17.697190 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 11:44:17.697389 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 11:44:17.697569 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":264,"threads_started":1}
I20260808 11:44:17.699992 21430 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44795
I20260808 11:44:17.700206 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 21256
I20260808 11:44:17.700320 21430 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:17.700397 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:17.700968 13945 external_mini_cluster.cc:1428] Setting key 73b41d38fcb221cdc4762764ed03d55a
I20260808 11:44:17.701000 21430 heartbeater.cc:511] Master 127.13.158.126:44795 requested a full tablet report, sending...
I20260808 11:44:17.702591 20785 ts_manager.cc:194] Registered new tserver with Master: 54fea3684ac04c4ab605633f84d036e2 (127.13.158.67:37019)
I20260808 11:44:17.704125 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:44:17.711304 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.708733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.13.158.126:44795)
0808 11:44:17.708889 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.708894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.709391 (+ 497us) server_negotiation.cc:407] Connection header received
0808 11:44:17.709596 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.709600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.709648 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.709753 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.710604 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.710610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.710614 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.710631 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.710669 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.710680 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.710865 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.710988 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.711049 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:44:17.715070 20786 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:44:17.721670 20786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:17.751659 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.735639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33981 (local address 127.0.0.1:36870)
0808 11:44:17.736407 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:17.736455 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:17.736633 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:17.748337 (+ 11704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:17.748349 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:17.748474 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:17.749575 (+ 1101us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:44:17.749578 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:17.749607 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:44:17.749697 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:17.751051 (+ 1354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:17.751061 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:17.751067 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:44:17.751195 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 11:44:17.751273 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":1480,"threads_started":1}
I20260808 11:44:17.753078 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.736074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.13.158.65:33981)
0808 11:44:17.747272 (+ 11198us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.747285 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.747331 (+ 46us) server_negotiation.cc:407] Connection header received
0808 11:44:17.747443 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.747456 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.747766 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.749883 (+ 2117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.750748 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.750777 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.750792 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.750819 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.750880 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.750917 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.752550 (+ 1633us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.752634 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.752736 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10945,"thread_start_us":11856,"threads_started":1}
I20260808 11:44:17.754061 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.736550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.13.158.67:37019)
0808 11:44:17.737247 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.737266 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.749399 (+ 12133us) server_negotiation.cc:407] Connection header received
0808 11:44:17.749484 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.749491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.749735 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.749890 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.752433 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.752441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.752453 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.752472 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.752525 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.752544 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.752724 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.753541 (+ 817us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.753647 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":382,"threads_started":1}
I20260808 11:44:17.754825 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.736010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37019 (local address 127.0.0.1:33730)
0808 11:44:17.749084 (+ 13074us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:17.749105 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:17.749240 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:17.751207 (+ 1967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:17.751215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:17.751250 (+ 35us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:17.752135 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:44:17.752141 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:17.752165 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:44:17.752228 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:17.752788 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:17.752793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:17.752800 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 11:44:17.752951 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 11:44:17.753010 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12880,"threads_started":1}
I20260808 11:44:17.756350 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.738238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:33783 (local address 127.0.0.1:40310)
0808 11:44:17.752239 (+ 14001us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:17.752277 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:17.752368 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:17.752855 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:17.752860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:17.752913 (+ 53us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:17.754541 (+ 1628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:44:17.754544 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:17.754559 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:44:17.754626 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:17.755924 (+ 1298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:17.755928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:17.755934 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:44:17.756044 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 11:44:17.756092 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13912,"thread_start_us":10410,"threads_started":1}
I20260808 11:44:17.756515 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.738653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.13.158.66:33783)
0808 11:44:17.749080 (+ 10427us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.749089 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.752364 (+ 3275us) server_negotiation.cc:407] Connection header received
0808 11:44:17.752468 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.752475 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.752675 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.753236 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.754845 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.754853 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.754865 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.754899 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.754988 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.755010 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.755157 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.756104 (+ 947us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.756202 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9756,"thread_start_us":9608,"threads_started":1}
I20260808 11:44:17.757449 20952 tablet_service.cc:1467] Processing CreateTablet for tablet a12b986a38344470829f6a39969eb462 (DEFAULT_TABLE table=test-table [id=7cb23d5d5b3a4a1e976d8af72aac5adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:17.758085 21340 tablet_service.cc:1467] Processing CreateTablet for tablet a12b986a38344470829f6a39969eb462 (DEFAULT_TABLE table=test-table [id=7cb23d5d5b3a4a1e976d8af72aac5adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:17.758543 20952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12b986a38344470829f6a39969eb462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:17.759294 21340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12b986a38344470829f6a39969eb462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:17.762292 21142 tablet_service.cc:1467] Processing CreateTablet for tablet a12b986a38344470829f6a39969eb462 (DEFAULT_TABLE table=test-table [id=7cb23d5d5b3a4a1e976d8af72aac5adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:17.763578 21142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12b986a38344470829f6a39969eb462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:17.770277 21455 tablet_bootstrap.cc:492] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: Bootstrap starting.
I20260808 11:44:17.773378 21456 tablet_bootstrap.cc:492] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Bootstrap starting.
I20260808 11:44:17.774940 21455 tablet_bootstrap.cc:654] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:17.776211 21455 log.cc:824] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:17.776795 21457 tablet_bootstrap.cc:492] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: Bootstrap starting.
I20260808 11:44:17.776907 21456 tablet_bootstrap.cc:654] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:17.778270 21456 log.cc:824] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:17.780376 21457 tablet_bootstrap.cc:654] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:17.781211 21456 tablet_bootstrap.cc:492] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: No bootstrap required, opened a new log
I20260808 11:44:17.781473 21456 ts_tablet_manager.cc:1397] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260808 11:44:17.781592 21455 tablet_bootstrap.cc:492] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: No bootstrap required, opened a new log
I20260808 11:44:17.781566 21457 log.cc:824] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:17.781854 21455 ts_tablet_manager.cc:1397] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260808 11:44:17.784394 21457 tablet_bootstrap.cc:492] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: No bootstrap required, opened a new log
I20260808 11:44:17.784626 21457 ts_tablet_manager.cc:1397] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 11:44:17.789371 21456 raft_consensus.cc:348] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.789740 21456 raft_consensus.cc:374] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:17.789850 21456 raft_consensus.cc:729] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b278b3b19cab45e2b0140f631c15e980, State: Initialized, Role: FOLLOWER
I20260808 11:44:17.790432 21456 consensus_queue.cc:260] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.791575 21041 heartbeater.cc:503] Master 127.13.158.126:44795 was elected leader, sending a full tablet report...
I20260808 11:44:17.792107 21456 ts_tablet_manager.cc:1428] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260808 11:44:17.792197 21455 raft_consensus.cc:348] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.792626 21455 raft_consensus.cc:374] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:17.792733 21455 raft_consensus.cc:729] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fea3684ac04c4ab605633f84d036e2, State: Initialized, Role: FOLLOWER
I20260808 11:44:17.793321 21455 consensus_queue.cc:260] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.794399 21430 heartbeater.cc:503] Master 127.13.158.126:44795 was elected leader, sending a full tablet report...
I20260808 11:44:17.795017 21455 ts_tablet_manager.cc:1428] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
I20260808 11:44:17.795957 21457 raft_consensus.cc:348] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.796241 21457 raft_consensus.cc:374] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:17.796299 21457 raft_consensus.cc:729] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b6721715c94200afa128fdda2c22bb, State: Initialized, Role: FOLLOWER
I20260808 11:44:17.796782 21457 consensus_queue.cc:260] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.797856 21248 heartbeater.cc:503] Master 127.13.158.126:44795 was elected leader, sending a full tablet report...
I20260808 11:44:17.798456 21457 ts_tablet_manager.cc:1428] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
W20260808 11:44:17.799108 21042 tablet.cc:2369] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:44:17.801880 21467 raft_consensus.cc:482] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:17.802150 21467 raft_consensus.cc:504] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.803937 21467 leader_election.cc:288] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b278b3b19cab45e2b0140f631c15e980 (127.13.158.65:33981), 54fea3684ac04c4ab605633f84d036e2 (127.13.158.67:37019)
I20260808 11:44:17.814672 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.804386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:42011 (local address 127.13.158.67:37019)
0808 11:44:17.804739 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.804749 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.812121 (+ 7372us) server_negotiation.cc:407] Connection header received
0808 11:44:17.812326 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.812331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.812396 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.812492 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.813951 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.813957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.813962 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.813983 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.814019 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.814034 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.814153 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.814338 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.814406 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260808 11:44:17.814867 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.804494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:49439 (local address 127.13.158.65:33981)
0808 11:44:17.804911 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.804917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.812867 (+ 7950us) server_negotiation.cc:407] Connection header received
0808 11:44:17.813038 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.813043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.813085 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.813222 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.814167 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.814174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.814179 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.814197 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.814232 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.814246 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.814346 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.814517 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.814571 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260808 11:44:17.814932 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.804186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37019 (local address 127.13.158.66:42011)
0808 11:44:17.811954 (+ 7768us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:17.811982 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:17.812155 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:17.812665 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:17.812670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:17.812695 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:17.813692 (+ 997us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:44:17.813696 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:17.813714 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:44:17.813759 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:17.814212 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:17.814215 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:17.814219 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 11:44:17.814380 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 11:44:17.814429 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7562,"thread_start_us":7344,"threads_started":1}
I20260808 11:44:17.815622 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.804267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33981 (local address 127.13.158.66:49439)
0808 11:44:17.812758 (+ 8491us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:17.812774 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:17.812908 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:17.813238 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:17.813243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:17.813264 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:17.813951 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:44:17.813954 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:17.813970 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:44:17.814014 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:17.814393 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:17.814399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:17.814404 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 11:44:17.815166 (+ 762us) client_negotiation.cc:240] Negotiation successful
0808 11:44:17.815217 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8374,"thread_start_us":8155,"threads_started":1}
I20260808 11:44:17.816011 21365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12b986a38344470829f6a39969eb462" candidate_uuid: "46b6721715c94200afa128fdda2c22bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fea3684ac04c4ab605633f84d036e2" is_pre_election: true
I20260808 11:44:17.816520 21365 raft_consensus.cc:2454] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46b6721715c94200afa128fdda2c22bb in term 0.
I20260808 11:44:17.816632 20978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12b986a38344470829f6a39969eb462" candidate_uuid: "46b6721715c94200afa128fdda2c22bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b278b3b19cab45e2b0140f631c15e980" is_pre_election: true
I20260808 11:44:17.817126 20978 raft_consensus.cc:2454] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46b6721715c94200afa128fdda2c22bb in term 0.
I20260808 11:44:17.817143 21067 leader_election.cc:302] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46b6721715c94200afa128fdda2c22bb, 54fea3684ac04c4ab605633f84d036e2; no voters:
I20260808 11:44:17.817472 21467 raft_consensus.cc:2781] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:17.817652 21467 raft_consensus.cc:482] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:17.817771 21467 raft_consensus.cc:3037] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:17.819984 21467 raft_consensus.cc:504] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.820626 21467 leader_election.cc:288] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [CANDIDATE]: Term 1 election: Requested vote from peers b278b3b19cab45e2b0140f631c15e980 (127.13.158.65:33981), 54fea3684ac04c4ab605633f84d036e2 (127.13.158.67:37019)
I20260808 11:44:17.821024 20978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12b986a38344470829f6a39969eb462" candidate_uuid: "46b6721715c94200afa128fdda2c22bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b278b3b19cab45e2b0140f631c15e980"
I20260808 11:44:17.821126 21365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12b986a38344470829f6a39969eb462" candidate_uuid: "46b6721715c94200afa128fdda2c22bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fea3684ac04c4ab605633f84d036e2"
I20260808 11:44:17.821242 20978 raft_consensus.cc:3037] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:17.821326 21365 raft_consensus.cc:3037] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:17.824548 21365 raft_consensus.cc:2454] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46b6721715c94200afa128fdda2c22bb in term 1.
I20260808 11:44:17.824549 20978 raft_consensus.cc:2454] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46b6721715c94200afa128fdda2c22bb in term 1.
I20260808 11:44:17.824954 21067 leader_election.cc:302] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46b6721715c94200afa128fdda2c22bb, 54fea3684ac04c4ab605633f84d036e2; no voters:
I20260808 11:44:17.825235 21467 raft_consensus.cc:2781] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:17.825713 21467 raft_consensus.cc:686] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 1 LEADER]: Becoming Leader. State: Replica: 46b6721715c94200afa128fdda2c22bb, State: Running, Role: LEADER
I20260808 11:44:17.826252 21467 consensus_queue.cc:237] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } }
I20260808 11:44:17.831414 20780 catalog_manager.cc:5495] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 46b6721715c94200afa128fdda2c22bb (127.13.158.66). New cstate: current_term: 1 leader_uuid: "46b6721715c94200afa128fdda2c22bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46b6721715c94200afa128fdda2c22bb" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33783 } health_report { overall_health: HEALTHY } } }
I20260808 11:44:17.856174 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.853704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.13.158.66:33783)
0808 11:44:17.853911 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.853917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.854025 (+ 108us) server_negotiation.cc:407] Connection header received
0808 11:44:17.854287 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.854291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.854371 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.854485 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.855492 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.855497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.855502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.855521 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.855564 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.855576 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.855706 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.855807 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.855867 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 11:44:17.864181 20978 raft_consensus.cc:1261] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 1 FOLLOWER]: Refusing update from remote peer 46b6721715c94200afa128fdda2c22bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:17.864220 21365 raft_consensus.cc:1261] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 1 FOLLOWER]: Refusing update from remote peer 46b6721715c94200afa128fdda2c22bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:17.865139 21471 consensus_queue.cc:1035] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fea3684ac04c4ab605633f84d036e2" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:17.865481 21467 consensus_queue.cc:1035] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b278b3b19cab45e2b0140f631c15e980" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:17.878013 21477 mvcc.cc:205] Tried to move back new op lower bound from 7316232019401732096 to 7316232019261235200. Current Snapshot: MvccSnapshot[applied={T|T < 7316232019401732096}]
I20260808 11:44:17.888572 20786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:17.888939 20786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:17.892930 20786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c80633a1dade49bf874baa789c44817b: Sending DeleteTablet for 3 replicas of tablet a12b986a38344470829f6a39969eb462
I20260808 11:44:17.894251 20952 tablet_service.cc:1514] Processing DeleteTablet for tablet a12b986a38344470829f6a39969eb462 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:17 UTC) from {username='slave'} at 127.0.0.1:36870
I20260808 11:44:17.894605 21142 tablet_service.cc:1514] Processing DeleteTablet for tablet a12b986a38344470829f6a39969eb462 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:17 UTC) from {username='slave'} at 127.0.0.1:40310
I20260808 11:44:17.894990 21340 tablet_service.cc:1514] Processing DeleteTablet for tablet a12b986a38344470829f6a39969eb462 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:17 UTC) from {username='slave'} at 127.0.0.1:33730
I20260808 11:44:17.895375 21487 tablet_replica.cc:330] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: stopping tablet replica
I20260808 11:44:17.895886 21487 raft_consensus.cc:2229] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:17.896487 21487 raft_consensus.cc:2258] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:17.896544 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 20861
I20260808 11:44:17.899170 21487 ts_tablet_manager.cc:1905] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:17.908396 21487 ts_tablet_manager.cc:1918] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:17.908708 21487 log.cc:1196] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/a12b986a38344470829f6a39969eb462
I20260808 11:44:17.909423 21487 ts_tablet_manager.cc:1939] T a12b986a38344470829f6a39969eb462 P b278b3b19cab45e2b0140f631c15e980: Deleting consensus metadata
I20260808 11:44:17.912444 20764 catalog_manager.cc:4841] TS b278b3b19cab45e2b0140f631c15e980 (127.13.158.65:33981): tablet a12b986a38344470829f6a39969eb462 (table test-table [id=7cb23d5d5b3a4a1e976d8af72aac5adb]) successfully deleted
I20260808 11:44:17.917222 21488 tablet_replica.cc:330] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: stopping tablet replica
I20260808 11:44:17.917702 21488 raft_consensus.cc:2229] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:44:17.918323 21488 raft_consensus.cc:2258] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:17.919626 21489 tablet_replica.cc:330] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: stopping tablet replica
I20260808 11:44:17.919981 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:17.899319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.13.158.65:33981)
0808 11:44:17.899484 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:17.899494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:17.915348 (+ 15854us) server_negotiation.cc:407] Connection header received
0808 11:44:17.915721 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:17.915730 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:17.915869 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:17.915975 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:17.919131 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:17.919139 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:17.919146 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:17.919172 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:17.919216 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:17.919236 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:17.919374 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:17.919631 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 11:44:17.919705 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 11:44:17.920071 21489 raft_consensus.cc:2229] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:17.920606 21489 raft_consensus.cc:2258] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:17.921128 21031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:17.924131 21488 ts_tablet_manager.cc:1905] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:17.929556 21488 ts_tablet_manager.cc:1918] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:17.929759 21488 log.cc:1196] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/a12b986a38344470829f6a39969eb462
I20260808 11:44:17.930368 21488 ts_tablet_manager.cc:1939] T a12b986a38344470829f6a39969eb462 P 46b6721715c94200afa128fdda2c22bb: Deleting consensus metadata
I20260808 11:44:17.930579 21489 ts_tablet_manager.cc:1905] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:17.932379 20765 catalog_manager.cc:4841] TS 46b6721715c94200afa128fdda2c22bb (127.13.158.66:33783): tablet a12b986a38344470829f6a39969eb462 (table test-table [id=7cb23d5d5b3a4a1e976d8af72aac5adb]) successfully deleted
I20260808 11:44:17.936028 21489 ts_tablet_manager.cc:1918] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:17.936213 21489 log.cc:1196] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/a12b986a38344470829f6a39969eb462
I20260808 11:44:17.936691 21489 ts_tablet_manager.cc:1939] T a12b986a38344470829f6a39969eb462 P 54fea3684ac04c4ab605633f84d036e2: Deleting consensus metadata
I20260808 11:44:17.938230 20763 catalog_manager.cc:4841] TS 54fea3684ac04c4ab605633f84d036e2 (127.13.158.67:37019): tablet a12b986a38344470829f6a39969eb462 (table test-table [id=7cb23d5d5b3a4a1e976d8af72aac5adb]) successfully deleted
I20260808 11:44:18.043493 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 20861
I20260808 11:44:18.072516 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 21046
I20260808 11:44:18.075681 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:18.073358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40334 (local address 127.13.158.66:33783)
0808 11:44:18.073530 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:18.073538 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:18.073562 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:44:18.073824 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:18.073832 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:18.073888 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:18.073997 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:18.074991 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:18.074997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:18.075003 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:18.075028 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:18.075061 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:18.075073 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:18.075219 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:18.075321 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 11:44:18.075378 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:44:18.076474 21241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:18.182793 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 21046
I20260808 11:44:18.206856 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 21256
I20260808 11:44:18.209575 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:18.207647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33744 (local address 127.13.158.67:37019)
0808 11:44:18.207838 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:18.207843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:18.207862 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:44:18.208022 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:18.208026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:18.208103 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:18.208209 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:18.208992 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:18.208997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:18.209000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:18.209015 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:18.209045 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:18.209057 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:18.209180 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:18.209216 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 11:44:18.209260 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:44:18.210253 21423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:18.297883 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 21256
I20260808 11:44:18.320742 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 20739
I20260808 11:44:18.321920 20824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:18.428305 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 20739
2026-08-08T11:44:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2657 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 11:44:18.481232 13945 mini_oidc.cc:152] Starting JWKS server
I20260808 11:44:18.484345 13945 webserver.cc:460] Webserver started at https://127.0.0.1:40201/ using document root <none> and password file <none>
I20260808 11:44:18.484720 13945 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 11:44:18.485450 13945 webserver.cc:460] Webserver started at http://127.0.0.1:34343/ using document root <none> and password file <none>
2026-08-08T11:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:18Z Disabled control of system clock
I20260808 11:44:18.520713 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:37267
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:37267
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40201/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/testchainca.cert with env {}
W20260808 11:44:18.851924 21504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:18.852229 21504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:18.852317 21504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:18.859321 21504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:18.859416 21504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:18.859453 21504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:18.859508 21504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:18.859550 21504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 11:44:18.859609 21504 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 11:44:18.859650 21504 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40201/jwks.json
I20260808 11:44:18.869316 21504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:37267
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:37267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40201/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:18.870527 21504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:18.871757 21504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:18.883671 21514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:18.883697 21515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:18.883627 21517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:18.884770 21504 server_base.cc:1034] running on GCE node
I20260808 11:44:18.885557 21504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:18.886965 21504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:18.888175 21504 hybrid_clock.cc:648] HybridClock initialized: now 1786189458888116 us; error 53 us; skew 500 ppm
I20260808 11:44:18.888459 21504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:44:18.890625 21504 webserver.cc:460] Webserver started at http://127.13.158.126:38377/ using document root <none> and password file <none>
I20260808 11:44:18.891333 21504 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:18.891429 21504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:18.891698 21504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:18.894500 21504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "5d0a18c3d1df453f8a1bb821ca961ae9"
format_stamp: "Formatted at 2026-08-08 11:44:18 on dist-test-slave-39l8"
server_key: "2194b7176c1a669299daf24da6ed13b9"
server_key_iv: "99beae298f39a64f5ce5fd49f592b798"
server_key_version: "encryptionkey@0"
I20260808 11:44:18.895366 21504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "5d0a18c3d1df453f8a1bb821ca961ae9"
format_stamp: "Formatted at 2026-08-08 11:44:18 on dist-test-slave-39l8"
server_key: "2194b7176c1a669299daf24da6ed13b9"
server_key_iv: "99beae298f39a64f5ce5fd49f592b798"
server_key_version: "encryptionkey@0"
I20260808 11:44:18.900345 21504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:44:18.903692 21527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:18.905061 21504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:18.905225 21504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "5d0a18c3d1df453f8a1bb821ca961ae9"
format_stamp: "Formatted at 2026-08-08 11:44:18 on dist-test-slave-39l8"
server_key: "2194b7176c1a669299daf24da6ed13b9"
server_key_iv: "99beae298f39a64f5ce5fd49f592b798"
server_key_version: "encryptionkey@0"
I20260808 11:44:18.905412 21504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:18.923204 21504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:18.923957 21504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:18.924226 21504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:18.943092 21504 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:37267
I20260808 11:44:18.943111 21599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:37267 every 8 connection(s)
I20260808 11:44:18.944437 21504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:18.948146 21600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:18.950336 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 21504
I20260808 11:44:18.950551 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:18.951236 13945 external_mini_cluster.cc:1428] Setting key 0bbe9d3d46304cb8b3f0d8678cc73993
I20260808 11:44:18.959753 21600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9: Bootstrap starting.
I20260808 11:44:18.963384 21600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:18.964537 21600 log.cc:824] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:18.965019 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:18.953161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.13.158.126:37267)
0808 11:44:18.954135 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:18.954145 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:18.954187 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:44:18.955897 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:18.955919 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:18.956377 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:18.956732 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:18.957370 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:18.958629 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:18.960451 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:18.961091 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:18.961246 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:18.962624 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:18.962635 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:18.962647 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:18.963189 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:18.963289 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:18.963311 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:18.963397 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:18.963652 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 11:44:18.964007 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":377,"threads_started":1}
I20260808 11:44:18.967589 21600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9: No bootstrap required, opened a new log
I20260808 11:44:18.968834 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:44:18.975190 21600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0a18c3d1df453f8a1bb821ca961ae9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37267 } }
I20260808 11:44:18.975557 21600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:18.975661 21600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d0a18c3d1df453f8a1bb821ca961ae9, State: Initialized, Role: FOLLOWER
I20260808 11:44:18.976255 21600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0a18c3d1df453f8a1bb821ca961ae9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37267 } }
I20260808 11:44:18.976490 21600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:18.976636 21600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:18.976809 21600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:18.979341 21600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0a18c3d1df453f8a1bb821ca961ae9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37267 } }
I20260808 11:44:18.979846 21600 leader_election.cc:302] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d0a18c3d1df453f8a1bb821ca961ae9; no voters:
I20260808 11:44:18.980399 21600 leader_election.cc:288] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:18.980576 21610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:18.981367 21610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [term 1 LEADER]: Becoming Leader. State: Replica: 5d0a18c3d1df453f8a1bb821ca961ae9, State: Running, Role: LEADER
I20260808 11:44:18.981914 21610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0a18c3d1df453f8a1bb821ca961ae9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37267 } }
I20260808 11:44:18.982456 21600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:18.984871 21611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d0a18c3d1df453f8a1bb821ca961ae9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0a18c3d1df453f8a1bb821ca961ae9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37267 } } }
I20260808 11:44:18.985277 21611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:18.985884 21612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d0a18c3d1df453f8a1bb821ca961ae9. Latest consensus state: current_term: 1 leader_uuid: "5d0a18c3d1df453f8a1bb821ca961ae9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0a18c3d1df453f8a1bb821ca961ae9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37267 } } }
I20260808 11:44:18.986277 21612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:18.988401 21618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:18.994019 21618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:19.003084 21618 catalog_manager.cc:1347] Generated new cluster ID: 84595eeb5f194305bb3f6edd369a909f
I20260808 11:44:19.003238 21618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:19.013279 21618 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:19.014484 21618 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:19.023852 21618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d0a18c3d1df453f8a1bb821ca961ae9: Generated new TSK 0
I20260808 11:44:19.024621 21618 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 11:44:19.068559 21642 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 11:44:19.068737 21642 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 11:44:19.069515 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:19.041386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.13.158.126:37267)
0808 11:44:19.041622 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:19.041632 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:19.042109 (+ 477us) server_negotiation.cc:407] Connection header received
0808 11:44:19.042438 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:19.042442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:19.042511 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:19.042628 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0808 11:44:19.043277 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.043934 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.046043 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.046444 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.046531 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:19.047120 (+ 589us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 11:44:19.068971 (+ 21851us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40201/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 11:44:19.069294 (+ 323us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40201/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 11:44:19.069677 21608 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40201/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 11:44:19.071552 21640 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:19.041291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:37267 (local address 127.0.0.1:35972)
0808 11:44:19.041977 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:19.041998 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:19.042216 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:19.042687 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:19.042693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:19.042727 (+ 34us) client_negotiation.cc:189] Negotiated authn=JWT
0808 11:44:19.043138 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:19.043151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.044076 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.044081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:19.045751 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:19.045767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.046618 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.046627 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:19.046818 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:19.046896 (+ 78us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 11:44:19.069292 (+ 22396us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40201/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 11:44:19.069651 (+ 359us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.158.126:37267: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40201/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":339,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.158.126:37267: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40201/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 11:44:19.073526 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 21504
I20260808 11:44:19.074687 21594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:19.166945 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 21504
2026-08-08T11:44:19Z chronyd exiting
I20260808 11:44:19.225023 13945 test_util.cc:175] -----------------------------------------------
I20260808 11:44:19.225198 13945 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189414476033-13945-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (748 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 11:44:19.229058 13945 mini_oidc.cc:152] Starting JWKS server
I20260808 11:44:19.230914 13945 webserver.cc:460] Webserver started at https://127.0.0.1:33409/ using document root <none> and password file <none>
I20260808 11:44:19.231161 13945 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 11:44:19.231781 13945 webserver.cc:460] Webserver started at http://127.0.0.1:38365/ using document root <none> and password file <none>
2026-08-08T11:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:19Z Disabled control of system clock
I20260808 11:44:19.262940 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36333
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:46423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:36333
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33409/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/testchainca.cert with env {}
W20260808 11:44:19.589344 21653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:19.589722 21653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:19.589886 21653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:19.597172 21653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:19.597309 21653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:19.597384 21653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:19.597492 21653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:19.597568 21653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 11:44:19.597664 21653 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 11:44:19.597738 21653 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33409/jwks.json
I20260808 11:44:19.608798 21653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:36333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33409/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:19.610289 21653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:19.611675 21653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:19.623715 21659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:19.623687 21662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:19.623764 21660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:19.624560 21653 server_base.cc:1034] running on GCE node
I20260808 11:44:19.625361 21653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:19.626763 21653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:19.627982 21653 hybrid_clock.cc:648] HybridClock initialized: now 1786189459627940 us; error 38 us; skew 500 ppm
I20260808 11:44:19.628340 21653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:44:19.635116 21653 webserver.cc:460] Webserver started at http://127.13.158.126:40891/ using document root <none> and password file <none>
I20260808 11:44:19.635882 21653 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:19.636019 21653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:19.636365 21653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:19.639421 21653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "b340ecd703194c8488b7452abf56afef"
format_stamp: "Formatted at 2026-08-08 11:44:19 on dist-test-slave-39l8"
server_key: "00059eb8aa13ea8987a57da833bc34f7"
server_key_iv: "aca775b69a6278a06c08d44e342e4f5b"
server_key_version: "encryptionkey@0"
I20260808 11:44:19.640254 21653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "b340ecd703194c8488b7452abf56afef"
format_stamp: "Formatted at 2026-08-08 11:44:19 on dist-test-slave-39l8"
server_key: "00059eb8aa13ea8987a57da833bc34f7"
server_key_iv: "aca775b69a6278a06c08d44e342e4f5b"
server_key_version: "encryptionkey@0"
I20260808 11:44:19.645535 21653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 11:44:19.648960 21671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:19.650360 21653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:44:19.650547 21653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "b340ecd703194c8488b7452abf56afef"
format_stamp: "Formatted at 2026-08-08 11:44:19 on dist-test-slave-39l8"
server_key: "00059eb8aa13ea8987a57da833bc34f7"
server_key_iv: "aca775b69a6278a06c08d44e342e4f5b"
server_key_version: "encryptionkey@0"
I20260808 11:44:19.650786 21653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:19.660988 21653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:19.661775 21653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:19.662083 21653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:19.679821 21653 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:36333
I20260808 11:44:19.679843 21747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:36333 every 8 connection(s)
I20260808 11:44:19.681253 21653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:19.684933 21748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:19.691653 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 21653
I20260808 11:44:19.691844 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:19.692507 13945 external_mini_cluster.cc:1428] Setting key 2a2fb4928039c0a3ad8f578219961edd
I20260808 11:44:19.695047 21748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef: Bootstrap starting.
I20260808 11:44:19.698280 21748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:19.699458 21748 log.cc:824] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:19.702438 21748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef: No bootstrap required, opened a new log
I20260808 11:44:19.705595 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:19.694296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.13.158.126:36333)
0808 11:44:19.695273 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:19.695283 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:19.695324 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:44:19.696911 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:19.696931 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:19.697369 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:19.697671 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:19.698345 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.699812 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.701429 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.702067 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.702212 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:19.703428 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:19.703435 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:19.703447 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:19.703944 (+ 497us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:19.704025 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:19.704045 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:19.704141 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:19.704375 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 11:44:19.704694 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":354,"threads_started":1}
I20260808 11:44:19.709686 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:44:19.711699 21748 raft_consensus.cc:348] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b340ecd703194c8488b7452abf56afef" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36333 } }
I20260808 11:44:19.712155 21748 raft_consensus.cc:374] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:19.712271 21748 raft_consensus.cc:729] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b340ecd703194c8488b7452abf56afef, State: Initialized, Role: FOLLOWER
I20260808 11:44:19.712949 21748 consensus_queue.cc:260] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b340ecd703194c8488b7452abf56afef" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36333 } }
I20260808 11:44:19.713229 21748 raft_consensus.cc:388] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:19.713387 21748 raft_consensus.cc:482] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:19.713568 21748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:19.715826 21748 raft_consensus.cc:504] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b340ecd703194c8488b7452abf56afef" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36333 } }
I20260808 11:44:19.716303 21748 leader_election.cc:302] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b340ecd703194c8488b7452abf56afef; no voters:
I20260808 11:44:19.716833 21748 leader_election.cc:288] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:19.716964 21754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:19.718290 21754 raft_consensus.cc:686] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [term 1 LEADER]: Becoming Leader. State: Replica: b340ecd703194c8488b7452abf56afef, State: Running, Role: LEADER
I20260808 11:44:19.718824 21748 sys_catalog.cc:564] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:19.718799 21754 consensus_queue.cc:237] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b340ecd703194c8488b7452abf56afef" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36333 } }
I20260808 11:44:19.721424 21756 sys_catalog.cc:455] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [sys.catalog]: SysCatalogTable state changed. Reason: New leader b340ecd703194c8488b7452abf56afef. Latest consensus state: current_term: 1 leader_uuid: "b340ecd703194c8488b7452abf56afef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b340ecd703194c8488b7452abf56afef" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36333 } } }
I20260808 11:44:19.721434 21755 sys_catalog.cc:455] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b340ecd703194c8488b7452abf56afef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b340ecd703194c8488b7452abf56afef" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36333 } } }
I20260808 11:44:19.721830 21755 sys_catalog.cc:458] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:19.721830 21756 sys_catalog.cc:458] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:19.723379 21761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:19.730176 21761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:19.740098 21761 catalog_manager.cc:1347] Generated new cluster ID: beb3674eb4f943588e55793a4b07c2f6
I20260808 11:44:19.740218 21761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:19.757689 21761 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:19.759526 21761 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:19.778553 21761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b340ecd703194c8488b7452abf56afef: Generated new TSK 0
I20260808 11:44:19.779657 21761 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 11:44:19.826975 21787 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 11:44:19.827111 21787 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 11:44:19.828029 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:19.800611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.13.158.126:36333)
0808 11:44:19.800843 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:19.800852 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:19.801334 (+ 482us) server_negotiation.cc:407] Connection header received
0808 11:44:19.801558 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:19.801561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:19.801636 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:19.801793 (+ 157us) server_negotiation.cc:226] Negotiated authn=JWT
0808 11:44:19.802298 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.803077 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.805110 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.805516 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.805608 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:19.806138 (+ 530us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 11:44:19.827494 (+ 21356us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33409/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 11:44:19.827826 (+ 332us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33409/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 11:44:19.828169 21751 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33409/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 11:44:19.828459 21786 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:19.800594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36333 (local address 127.0.0.1:46910)
0808 11:44:19.801217 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:19.801244 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:19.801398 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:19.801812 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:19.801820 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:19.801833 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0808 11:44:19.802120 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:19.802134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.803249 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.803255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:19.804972 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:19.804986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:19.805670 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:19.805677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:19.805860 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:19.805904 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 11:44:19.827738 (+ 21834us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33409/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 11:44:19.828036 (+ 298us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.158.126:36333: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33409/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":335,"threads_started":1}
I20260808 11:44:19.829540 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 21653
I20260808 11:44:19.830835 21742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:19.922408 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 21653
2026-08-08T11:44:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (758 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 11:44:19.987457 13945 mini_oidc.cc:152] Starting JWKS server
I20260808 11:44:19.989354 13945 webserver.cc:460] Webserver started at https://127.0.0.1:43423/ using document root <none> and password file <none>
I20260808 11:44:19.989544 13945 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 11:44:19.990100 13945 webserver.cc:460] Webserver started at http://127.0.0.1:35453/ using document root <none> and password file <none>
2026-08-08T11:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:20Z Disabled control of system clock
I20260808 11:44:20.022763 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:32931
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:41647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:32931
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43423/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 11:44:20.354672 21794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:20.354986 21794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:20.355154 21794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:20.362123 21794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:20.362253 21794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:20.362360 21794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:20.362466 21794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:20.362546 21794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 11:44:20.362643 21794 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 11:44:20.362730 21794 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43423/jwks.json
I20260808 11:44:20.372058 21794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:41647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:32931
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:32931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43423/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:20.373437 21794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:20.375290 21794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:20.387161 21799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:20.387126 21805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:20.387081 21800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:20.387955 21794 server_base.cc:1034] running on GCE node
I20260808 11:44:20.388754 21794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:20.390141 21794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:20.391368 21794 hybrid_clock.cc:648] HybridClock initialized: now 1786189460391322 us; error 37 us; skew 500 ppm
I20260808 11:44:20.391705 21794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:44:20.393904 21794 webserver.cc:460] Webserver started at http://127.13.158.126:39507/ using document root <none> and password file <none>
I20260808 11:44:20.394593 21794 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:20.394735 21794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:20.395071 21794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:20.397910 21794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "c3dce51757b343a7b73e732ab38d1198"
format_stamp: "Formatted at 2026-08-08 11:44:20 on dist-test-slave-39l8"
server_key: "93745bae129a17823537f15c022dbe55"
server_key_iv: "d8c0eb0d0219f5b1ed2ee4038094cc97"
server_key_version: "encryptionkey@0"
I20260808 11:44:20.398749 21794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "c3dce51757b343a7b73e732ab38d1198"
format_stamp: "Formatted at 2026-08-08 11:44:20 on dist-test-slave-39l8"
server_key: "93745bae129a17823537f15c022dbe55"
server_key_iv: "d8c0eb0d0219f5b1ed2ee4038094cc97"
server_key_version: "encryptionkey@0"
I20260808 11:44:20.404212 21794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 11:44:20.407822 21813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:20.409089 21794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260808 11:44:20.409293 21794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c3dce51757b343a7b73e732ab38d1198"
format_stamp: "Formatted at 2026-08-08 11:44:20 on dist-test-slave-39l8"
server_key: "93745bae129a17823537f15c022dbe55"
server_key_iv: "d8c0eb0d0219f5b1ed2ee4038094cc97"
server_key_version: "encryptionkey@0"
I20260808 11:44:20.409512 21794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:20.425768 21794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:20.426618 21794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:20.426975 21794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:20.445333 21794 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:32931
I20260808 11:44:20.445339 21883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:32931 every 8 connection(s)
I20260808 11:44:20.446693 21794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:20.450291 21884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:20.455179 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 21794
I20260808 11:44:20.455404 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:20.455999 13945 external_mini_cluster.cc:1428] Setting key b95e718438b03da81f1ddb762807947f
I20260808 11:44:20.461274 21884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198: Bootstrap starting.
I20260808 11:44:20.464425 21884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:20.465560 21884 log.cc:824] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:20.468457 21884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198: No bootstrap required, opened a new log
I20260808 11:44:20.469260 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:20.457862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36984 (local address 127.13.158.126:32931)
0808 11:44:20.458923 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:20.458934 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:20.458975 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:44:20.460633 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:20.460656 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:20.461137 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:20.461515 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:20.462085 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:20.463478 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:20.464884 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:20.465578 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:20.465774 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:20.466991 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:20.467005 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:20.467026 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:20.467522 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:20.467602 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:20.467621 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:20.467752 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:20.467946 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 11:44:20.468241 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":348,"threads_started":1}
I20260808 11:44:20.473230 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:44:20.476840 21884 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3dce51757b343a7b73e732ab38d1198" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32931 } }
I20260808 11:44:20.477187 21884 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:20.477294 21884 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3dce51757b343a7b73e732ab38d1198, State: Initialized, Role: FOLLOWER
I20260808 11:44:20.477814 21884 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3dce51757b343a7b73e732ab38d1198" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32931 } }
I20260808 11:44:20.478051 21884 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:20.478204 21884 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:20.478386 21884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:20.480810 21884 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3dce51757b343a7b73e732ab38d1198" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32931 } }
I20260808 11:44:20.481294 21884 leader_election.cc:302] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3dce51757b343a7b73e732ab38d1198; no voters:
I20260808 11:44:20.481865 21884 leader_election.cc:288] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:20.482004 21894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:20.482764 21894 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [term 1 LEADER]: Becoming Leader. State: Replica: c3dce51757b343a7b73e732ab38d1198, State: Running, Role: LEADER
I20260808 11:44:20.483332 21894 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3dce51757b343a7b73e732ab38d1198" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32931 } }
I20260808 11:44:20.483973 21884 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:20.486389 21895 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3dce51757b343a7b73e732ab38d1198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3dce51757b343a7b73e732ab38d1198" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32931 } } }
I20260808 11:44:20.486860 21895 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:20.487315 21896 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3dce51757b343a7b73e732ab38d1198. Latest consensus state: current_term: 1 leader_uuid: "c3dce51757b343a7b73e732ab38d1198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3dce51757b343a7b73e732ab38d1198" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32931 } } }
I20260808 11:44:20.487884 21896 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:20.490937 21905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:20.495047 21905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:20.502791 21905 catalog_manager.cc:1347] Generated new cluster ID: 10f5250237354c3abd01b0382a5d13ad
I20260808 11:44:20.502897 21905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:20.515910 21905 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:20.517059 21905 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:20.531580 21905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3dce51757b343a7b73e732ab38d1198: Generated new TSK 0
I20260808 11:44:20.532375 21905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:44:20.553371 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:20.545643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36996 (local address 127.13.158.126:32931)
0808 11:44:20.545818 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:20.545825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:20.546193 (+ 368us) server_negotiation.cc:407] Connection header received
0808 11:44:20.546433 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:20.546437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:20.546501 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:20.546706 (+ 205us) server_negotiation.cc:226] Negotiated authn=JWT
0808 11:44:20.547257 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:20.547971 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:20.549319 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:20.549749 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:20.549886 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:20.550364 (+ 478us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 11:44:20.552745 (+ 2381us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43423/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 11:44:20.553067 (+ 322us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36996: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43423/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 11:44:20.553495 21887 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36996: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43423/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 11:44:20.555608 21927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:20.545536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:32931 (local address 127.0.0.1:36996)
0808 11:44:20.546075 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:20.546109 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:20.546264 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:20.546762 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:20.546767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:20.546782 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0808 11:44:20.547094 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:20.547103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:20.548119 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:20.548127 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:20.549180 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:20.549190 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:20.549952 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:20.549959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:20.550090 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:20.550124 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 11:44:20.552963 (+ 2839us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43423/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 11:44:20.555117 (+ 2154us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.158.126:32931: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43423/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":1822,"thread_start_us":294,"threads_started":1}
I20260808 11:44:20.557404 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 21794
I20260808 11:44:20.558459 21879 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 11:44:20.567921 21928 webserver.cc:533] Webserver: sslize failed:
E20260808 11:44:20.568046 21928 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 11:44:20.652905 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 21794
2026-08-08T11:44:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (727 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T11:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:20Z Disabled control of system clock
I20260808 11:44:20.745621 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:41811
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:33857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:41811
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/testchainca.cert with env {}
W20260808 11:44:21.063310 21935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:21.063632 21935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:21.063792 21935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:21.070698 21935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:21.070902 21935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:21.070986 21935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:21.071115 21935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:21.071199 21935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 11:44:21.071293 21935 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 11:44:21.071365 21935 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 11:44:21.081138 21935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:41811
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:21.082408 21935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:21.083840 21935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:21.095826 21944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:21.095942 21941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:21.095857 21942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:21.096575 21935 server_base.cc:1034] running on GCE node
I20260808 11:44:21.097375 21935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:21.098984 21935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:21.100229 21935 hybrid_clock.cc:648] HybridClock initialized: now 1786189461100188 us; error 80 us; skew 500 ppm
I20260808 11:44:21.100574 21935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:44:21.102891 21935 webserver.cc:460] Webserver started at http://127.13.158.126:36889/ using document root <none> and password file <none>
I20260808 11:44:21.103606 21935 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:21.103737 21935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:21.104081 21935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:21.106945 21935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "c0861b96c339408e8c83de016d1cf79f"
format_stamp: "Formatted at 2026-08-08 11:44:21 on dist-test-slave-39l8"
server_key: "0c95d4796d9fe8ccc90bee2f1e97563e"
server_key_iv: "92f736009429ddd06608103fe511c788"
server_key_version: "encryptionkey@0"
I20260808 11:44:21.107738 21935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "c0861b96c339408e8c83de016d1cf79f"
format_stamp: "Formatted at 2026-08-08 11:44:21 on dist-test-slave-39l8"
server_key: "0c95d4796d9fe8ccc90bee2f1e97563e"
server_key_iv: "92f736009429ddd06608103fe511c788"
server_key_version: "encryptionkey@0"
I20260808 11:44:21.112712 21935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:44:21.116117 21951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:21.117358 21935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:44:21.117540 21935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c0861b96c339408e8c83de016d1cf79f"
format_stamp: "Formatted at 2026-08-08 11:44:21 on dist-test-slave-39l8"
server_key: "0c95d4796d9fe8ccc90bee2f1e97563e"
server_key_iv: "92f736009429ddd06608103fe511c788"
server_key_version: "encryptionkey@0"
I20260808 11:44:21.117769 21935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:21.134473 21935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:21.135383 21935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:21.135663 21935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:21.153926 21935 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:41811
I20260808 11:44:21.153962 22021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:41811 every 8 connection(s)
I20260808 11:44:21.155336 21935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:21.159026 22022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:21.163872 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 21935
I20260808 11:44:21.164131 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:21.164757 13945 external_mini_cluster.cc:1428] Setting key 26bffe5347b5c2e6e321c40534bd7c14
I20260808 11:44:21.169930 22022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f: Bootstrap starting.
I20260808 11:44:21.173077 22022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:21.174116 22022 log.cc:824] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:21.176954 22022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f: No bootstrap required, opened a new log
I20260808 11:44:21.178335 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:21.166771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39270 (local address 127.13.158.126:41811)
0808 11:44:21.167861 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:21.167874 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:21.167934 (+ 60us) server_negotiation.cc:407] Connection header received
0808 11:44:21.169549 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:21.169575 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:21.170095 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:21.170496 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:21.171088 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:21.172535 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:21.173967 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:21.174882 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:21.175094 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:21.176174 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:21.176182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:21.176195 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:44:21.176706 (+ 511us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:21.176779 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:44:21.176799 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:21.176918 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:21.177145 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 11:44:21.177443 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":411,"threads_started":1}
I20260808 11:44:21.182101 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:44:21.185284 22022 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0861b96c339408e8c83de016d1cf79f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 41811 } }
I20260808 11:44:21.185662 22022 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:21.185770 22022 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0861b96c339408e8c83de016d1cf79f, State: Initialized, Role: FOLLOWER
I20260808 11:44:21.186331 22022 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0861b96c339408e8c83de016d1cf79f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 41811 } }
I20260808 11:44:21.186560 22022 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:21.186698 22022 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:21.186915 22022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:21.189329 22022 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0861b96c339408e8c83de016d1cf79f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 41811 } }
I20260808 11:44:21.189826 22022 leader_election.cc:302] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0861b96c339408e8c83de016d1cf79f; no voters:
I20260808 11:44:21.190402 22022 leader_election.cc:288] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:21.190601 22029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:21.191946 22029 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [term 1 LEADER]: Becoming Leader. State: Replica: c0861b96c339408e8c83de016d1cf79f, State: Running, Role: LEADER
I20260808 11:44:21.192448 22029 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0861b96c339408e8c83de016d1cf79f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 41811 } }
I20260808 11:44:21.192622 22022 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:21.195294 22031 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0861b96c339408e8c83de016d1cf79f. Latest consensus state: current_term: 1 leader_uuid: "c0861b96c339408e8c83de016d1cf79f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0861b96c339408e8c83de016d1cf79f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 41811 } } }
I20260808 11:44:21.196034 22031 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:21.195704 22030 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0861b96c339408e8c83de016d1cf79f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0861b96c339408e8c83de016d1cf79f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 41811 } } }
I20260808 11:44:21.196241 22030 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:21.196990 22040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:21.208029 22040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:21.217144 22040 catalog_manager.cc:1347] Generated new cluster ID: 760df631c08a4a29961d241af4f03289
I20260808 11:44:21.217264 22040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:21.232949 22040 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:21.234249 22040 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:21.249800 22040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0861b96c339408e8c83de016d1cf79f: Generated new TSK 0
I20260808 11:44:21.250758 22040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:44:21.348398 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:21.267264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.13.158.126:41811)
0808 11:44:21.267435 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:21.267441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:21.267822 (+ 381us) server_negotiation.cc:407] Connection header received
0808 11:44:21.268025 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:21.268030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:21.268113 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:21.268261 (+ 148us) server_negotiation.cc:226] Negotiated authn=JWT
0808 11:44:21.268646 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:21.269809 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:21.271195 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:21.271883 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:21.272022 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:21.272429 (+ 407us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 11:44:21.347631 (+ 75202us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 11:44:21.348046 (+ 415us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39274: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 11:44:21.348546 22025 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39274: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 11:44:21.348551 22060 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:21.267137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:41811 (local address 127.0.0.1:39274)
0808 11:44:21.267730 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:21.267745 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:21.267856 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:21.268282 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:21.268285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:21.268293 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 11:44:21.268519 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:21.268526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:21.269938 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:21.269942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:21.271026 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:21.271038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:21.272042 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:21.272046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:21.272172 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:21.272201 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 11:44:21.347969 (+ 75768us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 11:44:21.348225 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.158.126:41811: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":329,"threads_started":1}
I20260808 11:44:21.349895 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 21935
I20260808 11:44:21.351320 22017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:21.450261 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 21935
2026-08-08T11:44:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (776 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (415 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (412 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:22 dist-test-slave-39l8 krb5kdc[22081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:22 dist-test-slave-39l8 krb5kdc[22081](info): set up 2 sockets
Aug 08 11:44:22 dist-test-slave-39l8 krb5kdc[22081](info): commencing operation
krb5kdc: starting...
W20260808 11:44:23.627151 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:23 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:44:23.854565 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43387
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:24.179809 22097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:24.180110 22097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:24.180269 22097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:24.188221 22097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:24.188468 22097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:24.188588 22097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:24.188704 22097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:24.188804 22097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:24.198890 22097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:24.200142 22097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:24.201402 22097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:24.212694 22106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:24.212771 22102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:24.212766 22103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:24.213709 22097 server_base.cc:1034] running on GCE node
I20260808 11:44:24.214380 22097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:24.215669 22097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:24.216884 22097 hybrid_clock.cc:648] HybridClock initialized: now 1786189464216845 us; error 47 us; skew 500 ppm
I20260808 11:44:24.217190 22097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:24 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189464, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:24.220203 22097 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:44:24.221599 22097 webserver.cc:460] Webserver started at http://127.13.158.126:34739/ using document root <none> and password file <none>
I20260808 11:44:24.222237 22097 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:24.222379 22097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:24.222764 22097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:24.225546 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "75bfcecd4a184d8e8407696e23f785ee"
format_stamp: "Formatted at 2026-08-08 11:44:24 on dist-test-slave-39l8"
server_key: "d27e71d45fddba3f42ef02d9d8d50371"
server_key_iv: "1a29353b42db384db791511d2bacbd59"
server_key_version: "encryptionkey@0"
I20260808 11:44:24.226301 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "75bfcecd4a184d8e8407696e23f785ee"
format_stamp: "Formatted at 2026-08-08 11:44:24 on dist-test-slave-39l8"
server_key: "d27e71d45fddba3f42ef02d9d8d50371"
server_key_iv: "1a29353b42db384db791511d2bacbd59"
server_key_version: "encryptionkey@0"
I20260808 11:44:24.231168 22097 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 11:44:24.234454 22116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:24.235818 22097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:24.235993 22097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "75bfcecd4a184d8e8407696e23f785ee"
format_stamp: "Formatted at 2026-08-08 11:44:24 on dist-test-slave-39l8"
server_key: "d27e71d45fddba3f42ef02d9d8d50371"
server_key_iv: "1a29353b42db384db791511d2bacbd59"
server_key_version: "encryptionkey@0"
I20260808 11:44:24.236193 22097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:24.263113 22097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:24.267217 22097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:24.267506 22097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:24.285656 22097 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:43387
I20260808 11:44:24.285687 22189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:43387 every 8 connection(s)
I20260808 11:44:24.287058 22097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:24.290571 22190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:24.294106 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 22097
I20260808 11:44:24.294349 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:24.295185 13945 external_mini_cluster.cc:1428] Setting key f8545bfe75f7901568c528f3f2ff295b
I20260808 11:44:24.303583 22190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee: Bootstrap starting.
I20260808 11:44:24.307271 22190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:24.308638 22190 log.cc:824] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:44:24 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:24.311650 22190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee: No bootstrap required, opened a new log
I20260808 11:44:24.317876 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:24.297169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.13.158.126:43387)
0808 11:44:24.297907 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:24.297917 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:24.297955 (+ 38us) server_negotiation.cc:407] Connection header received
0808 11:44:24.299573 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:24.299597 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:24.299952 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:24.300258 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:24.302003 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:24.303286 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:24.305067 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:24.305716 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:24.305837 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:24.310751 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:24.310764 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:24.310782 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:24.310832 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:24.313942 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:24.314586 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:24.314592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:24.314601 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:24.314823 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:24.315199 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:24.315204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:24.315216 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:24.315725 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:24.316042 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:24.316499 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 11:44:24.316915 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":349,"threads_started":1}
I20260808 11:44:24.319916 22190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bfcecd4a184d8e8407696e23f785ee" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43387 } }
I20260808 11:44:24.320333 22190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:24.320444 22190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75bfcecd4a184d8e8407696e23f785ee, State: Initialized, Role: FOLLOWER
I20260808 11:44:24.321060 22190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bfcecd4a184d8e8407696e23f785ee" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43387 } }
I20260808 11:44:24.321266 22190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:24.321367 22190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:24.321506 22190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:24.323637 22190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bfcecd4a184d8e8407696e23f785ee" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43387 } }
I20260808 11:44:24.324051 22190 leader_election.cc:302] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75bfcecd4a184d8e8407696e23f785ee; no voters:
I20260808 11:44:24.324553 22190 leader_election.cc:288] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:24.324690 22195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:24.325412 22195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [term 1 LEADER]: Becoming Leader. State: Replica: 75bfcecd4a184d8e8407696e23f785ee, State: Running, Role: LEADER
I20260808 11:44:24.325949 22195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bfcecd4a184d8e8407696e23f785ee" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43387 } }
I20260808 11:44:24.326676 22190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:24.329217 22196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75bfcecd4a184d8e8407696e23f785ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bfcecd4a184d8e8407696e23f785ee" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43387 } } }
I20260808 11:44:24.329630 22196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:24.329197 22197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75bfcecd4a184d8e8407696e23f785ee. Latest consensus state: current_term: 1 leader_uuid: "75bfcecd4a184d8e8407696e23f785ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bfcecd4a184d8e8407696e23f785ee" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43387 } } }
I20260808 11:44:24.330065 22197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:24.333469 22205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:24.338876 22205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:24.347384 22205 catalog_manager.cc:1347] Generated new cluster ID: 89db5d48e9464231953330103e547bc5
I20260808 11:44:24.347489 22205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:24.361675 22205 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:24.362989 22205 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:24.373199 22205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75bfcecd4a184d8e8407696e23f785ee: Generated new TSK 0
I20260808 11:44:24.373935 22205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:44:24.435957 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43387
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:24.790872 22221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:24.791145 22221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:24.791239 22221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:24.798308 22221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:24.798413 22221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:24.798532 22221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:24.808539 22221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:24.809746 22221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:24.811053 22221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:24.824905 22229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:24.824872 22232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:24.824831 22230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:24.825744 22221 server_base.cc:1034] running on GCE node
I20260808 11:44:24.826314 22221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:24.827313 22221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:24.828529 22221 hybrid_clock.cc:648] HybridClock initialized: now 1786189464828484 us; error 52 us; skew 500 ppm
I20260808 11:44:24.828823 22221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:24 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189464, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:24.834906 22221 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:44:24.836396 22221 webserver.cc:460] Webserver started at http://127.13.158.65:42079/ using document root <none> and password file <none>
I20260808 11:44:24.837080 22221 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:24.837224 22221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:24.837529 22221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:24.840646 22221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "01bb739c04604fb8912f5d89cc2b3c19"
format_stamp: "Formatted at 2026-08-08 11:44:24 on dist-test-slave-39l8"
server_key: "9ab275297dac67ebe9f7807562c527e4"
server_key_iv: "f374157e1d8abba5d17b2ddd071c0204"
server_key_version: "encryptionkey@0"
I20260808 11:44:24.841396 22221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "01bb739c04604fb8912f5d89cc2b3c19"
format_stamp: "Formatted at 2026-08-08 11:44:24 on dist-test-slave-39l8"
server_key: "9ab275297dac67ebe9f7807562c527e4"
server_key_iv: "f374157e1d8abba5d17b2ddd071c0204"
server_key_version: "encryptionkey@0"
I20260808 11:44:24.846422 22221 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:44:24.849848 22244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:24.851153 22221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:24.851323 22221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "01bb739c04604fb8912f5d89cc2b3c19"
format_stamp: "Formatted at 2026-08-08 11:44:24 on dist-test-slave-39l8"
server_key: "9ab275297dac67ebe9f7807562c527e4"
server_key_iv: "f374157e1d8abba5d17b2ddd071c0204"
server_key_version: "encryptionkey@0"
I20260808 11:44:24.851550 22221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:24.867933 22221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:24.871974 22221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:24.872371 22221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:24.873451 22221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:24.874791 22221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:24.874890 22221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:24.875037 22221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:24.875128 22221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:24.908200 22221 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:34557
I20260808 11:44:24.908244 22401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:34557 every 8 connection(s)
I20260808 11:44:24.909435 22221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:24.917201 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 22221
I20260808 11:44:24.917395 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:44:24.918025 13945 external_mini_cluster.cc:1428] Setting key b0985f0357864dc1c3ddaa5f48ef0dce
Aug 08 11:44:24 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189464, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:24.929870 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:24.911949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:42581 (local address 127.13.158.126:43387)
0808 11:44:24.912420 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:24.912425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:24.913326 (+ 901us) server_negotiation.cc:407] Connection header received
0808 11:44:24.915138 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:24.915144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:24.915233 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:24.915346 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:24.916794 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:24.917591 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:24.919259 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:24.919770 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:24.919895 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:24.923264 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:24.923273 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:24.923281 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:24.923324 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:24.926804 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:24.927471 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:24.927478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:24.927484 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:24.927641 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:24.928082 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:24.928089 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:24.928094 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:24.928472 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:24.928591 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:24.929431 (+ 840us) server_negotiation.cc:299] Negotiation successful
0808 11:44:24.929605 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":261,"threads_started":1}
I20260808 11:44:24.931360 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:24.912447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43387 (local address 127.13.158.65:42581)
0808 11:44:24.913161 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:24.913198 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:24.914882 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:24.915576 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:24.915583 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:24.916070 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:24.916597 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:24.916613 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:24.917755 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:24.917759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:24.919131 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:24.919145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:24.920003 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:24.920009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:24.920186 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:24.920866 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:24.920891 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:24.923062 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:24.926946 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:24.926952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:24.926963 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:24.927351 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:24.927745 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:24.927748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:24.927750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:24.927973 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:24.928592 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:24.928602 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:24.929014 (+ 412us) client_negotiation.cc:769] Sending connection context
0808 11:44:24.929353 (+ 339us) client_negotiation.cc:240] Negotiation successful
0808 11:44:24.929922 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":287,"threads_started":1}
I20260808 11:44:24.932950 22402 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43387
I20260808 11:44:24.933250 22402 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:24.933995 22402 heartbeater.cc:511] Master 127.13.158.126:43387 requested a full tablet report, sending...
I20260808 11:44:24.936108 22141 ts_manager.cc:194] Registered new tserver with Master: 01bb739c04604fb8912f5d89cc2b3c19 (127.13.158.65:34557)
I20260808 11:44:24.937570 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:42581
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:44:24.969615 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43387
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:25.298045 22409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:25.298305 22409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:25.298451 22409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:25.305305 22409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:25.305394 22409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:25.305533 22409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:25.314806 22409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:25.315958 22409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:25.317189 22409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:25.329777 22415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:25.329694 22416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:25.329743 22418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:25.330771 22409 server_base.cc:1034] running on GCE node
I20260808 11:44:25.331419 22409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:25.332311 22409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:25.333541 22409 hybrid_clock.cc:648] HybridClock initialized: now 1786189465333483 us; error 48 us; skew 500 ppm
I20260808 11:44:25.333830 22409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:25 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189465, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:25.336870 22409 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:44:25.338201 22409 webserver.cc:460] Webserver started at http://127.13.158.66:44631/ using document root <none> and password file <none>
I20260808 11:44:25.338893 22409 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:25.339031 22409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:25.339327 22409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:25.342253 22409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "aee965641c8f4a489712f208b9f5ee53"
format_stamp: "Formatted at 2026-08-08 11:44:25 on dist-test-slave-39l8"
server_key: "343ea36d6e947df948a7fee9a581a7a7"
server_key_iv: "dbe33b896e6ff36e71a98adf1e5dc22f"
server_key_version: "encryptionkey@0"
I20260808 11:44:25.343077 22409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "aee965641c8f4a489712f208b9f5ee53"
format_stamp: "Formatted at 2026-08-08 11:44:25 on dist-test-slave-39l8"
server_key: "343ea36d6e947df948a7fee9a581a7a7"
server_key_iv: "dbe33b896e6ff36e71a98adf1e5dc22f"
server_key_version: "encryptionkey@0"
I20260808 11:44:25.347836 22409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:44:25.351073 22426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:25.352320 22409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:44:25.352488 22409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "aee965641c8f4a489712f208b9f5ee53"
format_stamp: "Formatted at 2026-08-08 11:44:25 on dist-test-slave-39l8"
server_key: "343ea36d6e947df948a7fee9a581a7a7"
server_key_iv: "dbe33b896e6ff36e71a98adf1e5dc22f"
server_key_version: "encryptionkey@0"
I20260808 11:44:25.352672 22409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:25.384212 22409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:25.387948 22409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:25.388240 22409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:25.389161 22409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:25.390514 22409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:25.390586 22409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:25.390702 22409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:25.390784 22409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 11:44:25.423460 22409 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:37183
I20260808 11:44:25.423537 22569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:37183 every 8 connection(s)
I20260808 11:44:25.424701 22409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:25.431537 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 22409
I20260808 11:44:25.431744 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:25.432240 13945 external_mini_cluster.cc:1428] Setting key 1e14894744be57d3628dd4c38fab8d8d
Aug 08 11:44:25 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189465, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:25.446187 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:25.427259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:41063 (local address 127.13.158.126:43387)
0808 11:44:25.427427 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:25.427431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:25.428618 (+ 1187us) server_negotiation.cc:407] Connection header received
0808 11:44:25.430415 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:25.430421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:25.430510 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:25.430618 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:25.432291 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:25.433182 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:25.435330 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:25.435738 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:25.435838 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:25.440077 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:25.440082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:25.440085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:25.440117 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:25.443150 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:25.443790 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:25.443812 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:25.443817 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:25.443952 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:25.444342 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:25.444349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:25.444354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:25.444722 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:25.444868 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:25.445501 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 11:44:25.445750 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:44:25.446806 22574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:25.427751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43387 (local address 127.13.158.66:41063)
0808 11:44:25.428459 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:25.428496 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:25.430153 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:25.430873 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:25.430881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:25.431377 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:25.432064 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:25.432083 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:25.433413 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:25.433418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:25.435192 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:25.435208 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:25.435970 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:25.435980 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:25.436296 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:25.437021 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:25.437059 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:25.439857 (+ 2798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:25.443274 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:25.443280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:25.443290 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:25.443650 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:25.444042 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:25.444045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:25.444047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:25.444228 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:25.444844 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:25.444849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:25.445124 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 11:44:25.445361 (+ 237us) client_negotiation.cc:240] Negotiation successful
0808 11:44:25.445701 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":306,"threads_started":1}
I20260808 11:44:25.448133 22572 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43387
I20260808 11:44:25.448408 22572 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:25.449115 22572 heartbeater.cc:511] Master 127.13.158.126:43387 requested a full tablet report, sending...
I20260808 11:44:25.450505 22141 ts_manager.cc:194] Registered new tserver with Master: aee965641c8f4a489712f208b9f5ee53 (127.13.158.66:37183)
I20260808 11:44:25.451352 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:41063
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:44:25.481981 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43387
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:25.809139 22579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:25.809425 22579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:25.809515 22579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:25.816887 22579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:25.816991 22579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:25.817150 22579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:25.827929 22579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:25.829218 22579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:25.830443 22579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:25.844002 22593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:25.844033 22589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:25.844956 22586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:25.845050 22579 server_base.cc:1034] running on GCE node
I20260808 11:44:25.845722 22579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:25.846603 22579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:25.847826 22579 hybrid_clock.cc:648] HybridClock initialized: now 1786189465847775 us; error 74 us; skew 500 ppm
I20260808 11:44:25.848131 22579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:25 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189465, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:25.851269 22579 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:44:25.852605 22579 webserver.cc:460] Webserver started at http://127.13.158.67:36343/ using document root <none> and password file <none>
I20260808 11:44:25.853269 22579 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:25.853404 22579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:25.853722 22579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:25.856673 22579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "3a4eb0735dae430697167a56a5633fe3"
format_stamp: "Formatted at 2026-08-08 11:44:25 on dist-test-slave-39l8"
server_key: "8431e47e8eaf4ffc34d865d33e0c47c7"
server_key_iv: "31c78b7542fc91bbba6a2af259bc1b41"
server_key_version: "encryptionkey@0"
I20260808 11:44:25.857445 22579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "3a4eb0735dae430697167a56a5633fe3"
format_stamp: "Formatted at 2026-08-08 11:44:25 on dist-test-slave-39l8"
server_key: "8431e47e8eaf4ffc34d865d33e0c47c7"
server_key_iv: "31c78b7542fc91bbba6a2af259bc1b41"
server_key_version: "encryptionkey@0"
I20260808 11:44:25.862217 22579 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260808 11:44:25.865525 22604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:25.866768 22579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:44:25.866935 22579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "3a4eb0735dae430697167a56a5633fe3"
format_stamp: "Formatted at 2026-08-08 11:44:25 on dist-test-slave-39l8"
server_key: "8431e47e8eaf4ffc34d865d33e0c47c7"
server_key_iv: "31c78b7542fc91bbba6a2af259bc1b41"
server_key_version: "encryptionkey@0"
I20260808 11:44:25.867136 22579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:25.886574 22579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:25.890805 22579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:25.891096 22579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:25.892062 22579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:25.893356 22579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:25.893432 22579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:25.893591 22579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:25.893647 22579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:25.927167 22579 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:40421
I20260808 11:44:25.927212 22749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:40421 every 8 connection(s)
I20260808 11:44:25.928372 22579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:25.932219 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 22579
I20260808 11:44:25.932410 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:25.933060 13945 external_mini_cluster.cc:1428] Setting key ae1bce54a48565d61ef24ff914266ded
I20260808 11:44:25.940382 22402 heartbeater.cc:503] Master 127.13.158.126:43387 was elected leader, sending a full tablet report...
Aug 08 11:44:25 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189465, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:25.948269 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:25.930914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:40893 (local address 127.13.158.126:43387)
0808 11:44:25.931137 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:25.931143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:25.932486 (+ 1343us) server_negotiation.cc:407] Connection header received
0808 11:44:25.934797 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:25.934801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:25.934883 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:25.934982 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:25.936631 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:25.937550 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:25.939064 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:25.939444 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:25.939536 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:25.942989 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:25.942994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:25.942997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:25.943027 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:25.945293 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:25.945908 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:25.945911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:25.945913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:25.946030 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:25.946410 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:25.946413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:25.946415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:25.946775 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:25.946882 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:25.947728 (+ 846us) server_negotiation.cc:299] Negotiation successful
0808 11:44:25.947945 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:44:25.949117 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:25.931381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43387 (local address 127.13.158.67:40893)
0808 11:44:25.932297 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:25.932355 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:25.934485 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:25.935280 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:25.935288 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:25.935760 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:25.936479 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:25.936496 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:25.937673 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:25.937696 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:25.938929 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:25.938940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:25.939617 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:25.939621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:25.939807 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:25.940544 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:25.940573 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:25.942801 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:25.945408 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:25.945414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:25.945424 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:25.945806 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:25.946112 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:25.946115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:25.946118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:25.946286 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:25.946888 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:25.946896 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:25.947304 (+ 408us) client_negotiation.cc:769] Sending connection context
0808 11:44:25.947605 (+ 301us) client_negotiation.cc:240] Negotiation successful
0808 11:44:25.948081 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":374,"threads_started":1}
I20260808 11:44:25.950415 22750 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43387
I20260808 11:44:25.950667 22750 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:25.951381 22750 heartbeater.cc:511] Master 127.13.158.126:43387 requested a full tablet report, sending...
I20260808 11:44:25.952652 22141 ts_manager.cc:194] Registered new tserver with Master: 3a4eb0735dae430697167a56a5633fe3 (127.13.158.67:40421)
I20260808 11:44:25.953384 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:40893
I20260808 11:44:25.960105 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/test-admin.keytab.
Aug 08 11:44:25 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:25.989444 13945 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 11:44:25.994551 22759 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:25.992207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.13.158.65:34557)
0808 11:44:25.992844 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:25.992852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:25.992878 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:44:25.992942 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:25.992948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:25.993147 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:25.993284 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:25.994165 (+ 881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49114: BlockingRecv error: recv got EOF from 127.0.0.1:49114 (error 108)
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":323,"threads_started":1}
W20260808 11:44:25.994596 13945 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5cc' of type FILE
Aug 08 11:44:25 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:25.996331 13945 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 11:44:25 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:25.997598 13945 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:44:26.011958 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:25.999167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.13.158.65:34557)
0808 11:44:25.999337 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:25.999344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:25.999483 (+ 139us) server_negotiation.cc:407] Connection header received
0808 11:44:25.999627 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:25.999630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:25.999682 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:25.999757 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:26.000654 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.001584 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.002765 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.003396 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.003491 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:26.006689 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:26.006700 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:26.006756 (+ 56us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:26.006824 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:26.009487 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.010104 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.010110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.010113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.010230 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.010514 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.010517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.010519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.010884 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:26.011097 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:26.011376 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 11:44:26.011659 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 11:44:26.019129 22759 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:26.017091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49124 (local address 127.13.158.65:34557)
0808 11:44:26.017277 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:26.017282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:26.017539 (+ 257us) server_negotiation.cc:407] Connection header received
0808 11:44:26.017733 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:26.017742 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:26.017829 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:26.017925 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:26.018898 (+ 973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49124: BlockingRecv error: recv got EOF from 127.0.0.1:49124 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 11:44:26.019979 13945 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5cc' of type FILE
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:26.021808 13945 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:26.023172 13945 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:44:26.037161 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:26.024404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.13.158.65:34557)
0808 11:44:26.024604 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:26.024611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:26.024882 (+ 271us) server_negotiation.cc:407] Connection header received
0808 11:44:26.025047 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:26.025052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:26.025116 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:26.025216 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:26.026127 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.026994 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.028250 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.028661 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.028798 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:26.032316 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:26.032323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:26.032328 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:26.032366 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:26.034817 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.035395 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.035401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.035405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.035523 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.035904 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.035909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.035912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.036157 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:26.036415 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:26.036684 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 11:44:26.036878 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 11:44:26.041455 22759 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:26.039848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.13.158.65:34557)
0808 11:44:26.040016 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:26.040022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:26.040243 (+ 221us) server_negotiation.cc:407] Connection header received
0808 11:44:26.040404 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:26.040409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:26.040481 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:26.040573 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:26.041231 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49156: BlockingRecv error: recv got EOF from 127.0.0.1:49156 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 11:44:26.041654 13945 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189414476033-13945-0/krb5kdc/krb5cc' of type FILE
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:26.043334 13945 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:26.044585 13945 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:44:26.058130 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:26.045740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.13.158.65:34557)
0808 11:44:26.045912 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:26.045919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:26.046209 (+ 290us) server_negotiation.cc:407] Connection header received
0808 11:44:26.046367 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:26.046372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:26.046431 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:26.046525 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:26.047458 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.048172 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.049366 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.049781 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.049910 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:26.053226 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:26.053231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:26.053234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:26.053264 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:26.055930 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.056546 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.056550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.056552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.056691 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.057062 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.057065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.057066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.057335 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:26.057558 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:26.057725 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 11:44:26.057874 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 11:44:26.059564 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 22221
I20260808 11:44:26.070096 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:26.060239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.13.158.65:34557)
0808 11:44:26.060372 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:26.060376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:26.060687 (+ 311us) server_negotiation.cc:407] Connection header received
0808 11:44:26.060796 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:26.060799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:26.060885 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:26.060957 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:26.061762 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.062375 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.063529 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.063918 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.064000 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:26.065719 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:26.065723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:26.065727 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:26.065754 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:26.068128 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.068654 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.068657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.068659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.068772 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.069099 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.069102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.069104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.069317 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:26.069494 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:26.069718 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 11:44:26.069857 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 11:44:26.070652 22393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:26.152150 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 22221
I20260808 11:44:26.175379 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 22409
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:44:26.190636 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:26.176290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59358 (local address 127.13.158.66:37183)
0808 11:44:26.176763 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:26.176770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:26.176815 (+ 45us) server_negotiation.cc:407] Connection header received
0808 11:44:26.176875 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:26.176881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:26.177083 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:26.177257 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:26.178274 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.179283 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.180600 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.181141 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.181275 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:26.185212 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:26.185222 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:26.185235 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:26.185265 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:26.187993 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.188620 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.188626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.188629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.188743 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.189145 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.189150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.189153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.189470 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:26.189772 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:26.190054 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 11:44:26.190306 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":234,"threads_started":1}
I20260808 11:44:26.191866 22565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:26.278105 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 22409
I20260808 11:44:26.309885 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 22579
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:44:26.328140 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:26.311116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.13.158.67:40421)
0808 11:44:26.311831 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:26.311837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:26.311865 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:44:26.311925 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:26.311931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:26.312151 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:26.312318 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:26.313464 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.314549 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.315971 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:26.316546 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:26.316690 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:26.321669 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:26.321681 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:26.321694 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:26.321732 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:26.324932 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.325872 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.325879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.325881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.326017 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:26.326480 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:26.326483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:26.326485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:26.326877 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:26.327107 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:26.327545 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 11:44:26.327747 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":383,"threads_started":1}
I20260808 11:44:26.329691 22744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:26.411700 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 22579
I20260808 11:44:26.435702 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 22097
I20260808 11:44:26.436950 22181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:26.537734 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 22097
2026-08-08T11:44:26Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4263 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22785](info): set up 2 sockets
Aug 08 11:44:26 dist-test-slave-39l8 krb5kdc[22785](info): commencing operation
krb5kdc: starting...
W20260808 11:44:27.812727 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:27 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.126.keytab.
Entry for principal oryx/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.126.keytab.
I20260808 11:44:28.032107 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:45849
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.126.keytab
--principal=oryx/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:45849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:28.352294 22809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:28.352614 22809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:28.352788 22809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:28.359652 22809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:28.359782 22809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:28.359865 22809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:28.359963 22809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:28.360029 22809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:28.369271 22809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:45849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.126.keytab
--principal=oryx/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:45849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:28.370555 22809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:28.371814 22809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:28.383344 22820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:28.383401 22817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:28.383312 22818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:28.384191 22809 server_base.cc:1034] running on GCE node
I20260808 11:44:28.384950 22809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:28.386238 22809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:28.387470 22809 hybrid_clock.cc:648] HybridClock initialized: now 1786189468387417 us; error 35 us; skew 500 ppm
I20260808 11:44:28.387804 22809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:28 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189468, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:28.390691 22809 init.cc:377] Logged in from keytab as oryx/127.13.158.126@KRBTEST.COM (short username oryx)
I20260808 11:44:28.392086 22809 webserver.cc:460] Webserver started at http://127.13.158.126:42783/ using document root <none> and password file <none>
I20260808 11:44:28.392707 22809 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:28.392807 22809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:28.393069 22809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:28.395936 22809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "560789189ca9482d8310a358e6abfd54"
format_stamp: "Formatted at 2026-08-08 11:44:28 on dist-test-slave-39l8"
server_key: "1590148e3ab13881b576e8368fd9e3dc"
server_key_iv: "c947b4d0f4ea35440f411ffe472608d6"
server_key_version: "encryptionkey@0"
I20260808 11:44:28.396651 22809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "560789189ca9482d8310a358e6abfd54"
format_stamp: "Formatted at 2026-08-08 11:44:28 on dist-test-slave-39l8"
server_key: "1590148e3ab13881b576e8368fd9e3dc"
server_key_iv: "c947b4d0f4ea35440f411ffe472608d6"
server_key_version: "encryptionkey@0"
I20260808 11:44:28.401578 22809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 11:44:28.404910 22827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:28.406173 22809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:44:28.406373 22809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "560789189ca9482d8310a358e6abfd54"
format_stamp: "Formatted at 2026-08-08 11:44:28 on dist-test-slave-39l8"
server_key: "1590148e3ab13881b576e8368fd9e3dc"
server_key_iv: "c947b4d0f4ea35440f411ffe472608d6"
server_key_version: "encryptionkey@0"
I20260808 11:44:28.406574 22809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:28.419950 22809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:28.424341 22809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:28.424635 22809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:28.442221 22809 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:45849
I20260808 11:44:28.442257 22897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:45849 every 8 connection(s)
I20260808 11:44:28.443694 22809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:28.447487 22898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:28.452961 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 22809
I20260808 11:44:28.453176 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:28.453840 13945 external_mini_cluster.cc:1428] Setting key 3fba3ea4109b12ab9f5cc21ca5f3c9f6
I20260808 11:44:28.458252 22898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: Bootstrap starting.
I20260808 11:44:28.461974 22898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:28.463263 22898 log.cc:824] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:28.465993 22898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: No bootstrap required, opened a new log
Aug 08 11:44:28 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:28.473398 22898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } }
I20260808 11:44:28.473778 22898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:28.473883 22898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560789189ca9482d8310a358e6abfd54, State: Initialized, Role: FOLLOWER
I20260808 11:44:28.474511 22898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } }
I20260808 11:44:28.474776 22898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:28.474924 22898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:28.475165 22898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:28.476203 22902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:28.455902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.13.158.126:45849)
0808 11:44:28.456733 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:28.456743 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:28.456784 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:44:28.458433 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:28.458454 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:28.458868 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:28.459159 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:28.461085 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:28.462386 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:28.464145 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:28.464804 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:28.464976 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:28.468967 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:28.468977 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:28.468991 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:28.469030 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:28.472118 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:28.472806 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:28.472813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:28.472819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:28.472976 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:28.473353 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:28.473357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:28.473359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:28.473999 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:28.474351 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:28.474764 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 11:44:28.475183 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":434,"threads_started":1}
I20260808 11:44:28.477936 22898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } }
I20260808 11:44:28.478415 22898 leader_election.cc:302] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 560789189ca9482d8310a358e6abfd54; no voters:
I20260808 11:44:28.479195 22898 leader_election.cc:288] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:28.479290 22904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:28.480110 22904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 1 LEADER]: Becoming Leader. State: Replica: 560789189ca9482d8310a358e6abfd54, State: Running, Role: LEADER
I20260808 11:44:28.480624 22904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } }
I20260808 11:44:28.481376 22898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:28.483385 22907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 560789189ca9482d8310a358e6abfd54. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } }
I20260808 11:44:28.483846 22907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:28.484424 22905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } }
I20260808 11:44:28.484800 22905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:28.488420 22914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:28.492957 22914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:28.501117 22914 catalog_manager.cc:1347] Generated new cluster ID: d7473c94efbd4a0085f2b819a32e48a1
I20260808 11:44:28.501227 22914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:28.531518 22914 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:28.532902 22914 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:28.542016 22914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: Generated new TSK 0
I20260808 11:44:28.542905 22914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab.
Entry for principal oryx/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab.
I20260808 11:44:28.606563 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45849
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab
--principal=oryx/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:28.942449 22934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:28.942795 22934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:28.943007 22934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:28.950362 22934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:28.950578 22934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:28.950824 22934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:28.960321 22934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab
--principal=oryx/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:28.961614 22934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:28.962987 22934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:28.976136 22940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:28.976109 22943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:28.976171 22941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:28.976967 22934 server_base.cc:1034] running on GCE node
I20260808 11:44:28.977556 22934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:28.978564 22934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:28.979779 22934 hybrid_clock.cc:648] HybridClock initialized: now 1786189468979724 us; error 51 us; skew 500 ppm
I20260808 11:44:28.980082 22934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:28 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189468, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:28.983520 22934 init.cc:377] Logged in from keytab as oryx/127.13.158.65@KRBTEST.COM (short username oryx)
I20260808 11:44:28.984843 22934 webserver.cc:460] Webserver started at http://127.13.158.65:43199/ using document root <none> and password file <none>
I20260808 11:44:28.985646 22934 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:28.985783 22934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:28.986079 22934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:28.989177 22934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "3f196183a33d49499432ed6622045abf"
format_stamp: "Formatted at 2026-08-08 11:44:28 on dist-test-slave-39l8"
server_key: "395fcd2df2ba463a0c1ee9c129cbe454"
server_key_iv: "9173820f47776cced8e4b7ba937959ce"
server_key_version: "encryptionkey@0"
I20260808 11:44:28.989996 22934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "3f196183a33d49499432ed6622045abf"
format_stamp: "Formatted at 2026-08-08 11:44:28 on dist-test-slave-39l8"
server_key: "395fcd2df2ba463a0c1ee9c129cbe454"
server_key_iv: "9173820f47776cced8e4b7ba937959ce"
server_key_version: "encryptionkey@0"
I20260808 11:44:28.994995 22934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:44:28.998229 22954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:28.999503 22934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:28.999671 22934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "3f196183a33d49499432ed6622045abf"
format_stamp: "Formatted at 2026-08-08 11:44:28 on dist-test-slave-39l8"
server_key: "395fcd2df2ba463a0c1ee9c129cbe454"
server_key_iv: "9173820f47776cced8e4b7ba937959ce"
server_key_version: "encryptionkey@0"
I20260808 11:44:28.999882 22934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:29.015448 22934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:29.019495 22934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:29.019790 22934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:29.020665 22934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:29.021924 22934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:29.022006 22934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:29.022176 22934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:29.022239 22934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:29.055186 22934 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:38839
I20260808 11:44:29.055263 23099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:38839 every 8 connection(s)
I20260808 11:44:29.056452 22934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:29.060335 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 22934
I20260808 11:44:29.060559 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:44:29.061396 13945 external_mini_cluster.cc:1428] Setting key 1375e707d8906c102634c3eb03e1ce7e
Aug 08 11:44:29 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189468, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:29.078729 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:29.058976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:33359 (local address 127.13.158.126:45849)
0808 11:44:29.059458 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:29.059462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:29.060401 (+ 939us) server_negotiation.cc:407] Connection header received
0808 11:44:29.062846 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:29.062851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:29.062930 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:29.063053 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:29.064939 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:29.065668 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:29.068209 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:29.068623 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:29.068738 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:29.072931 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:29.072936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:29.072939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:29.072971 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:29.075706 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:29.076421 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:29.076426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:29.076428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:29.076545 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:29.076991 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:29.076996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:29.077000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:29.077258 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:29.077412 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:29.078116 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 11:44:29.078336 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":254,"threads_started":1}
I20260808 11:44:29.079716 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:29.059402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.65:33359)
0808 11:44:29.060222 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:29.060273 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:29.062544 (+ 2271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:29.063292 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:29.063302 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:29.064002 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:29.064763 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:29.064790 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:29.065842 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:29.065849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:29.068035 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:29.068054 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:29.068881 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:29.068890 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:29.069236 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:29.070007 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:29.070041 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:29.072701 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:29.075849 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:29.075857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:29.075874 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:29.076286 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:29.076663 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:29.076667 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:29.076671 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:29.076869 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:29.077385 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:29.077393 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:29.077736 (+ 343us) client_negotiation.cc:769] Sending connection context
0808 11:44:29.077997 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 11:44:29.078522 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":277,"threads_started":1}
I20260808 11:44:29.081394 23101 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45849
I20260808 11:44:29.081759 23101 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:29.082536 23101 heartbeater.cc:511] Master 127.13.158.126:45849 requested a full tablet report, sending...
I20260808 11:44:29.085134 22851 ts_manager.cc:194] Registered new tserver with Master: 3f196183a33d49499432ed6622045abf (127.13.158.65:38839)
WARNING: no policy specified for oryx/127.13.158.66@KRBTEST.COM; defaulting to no policy
I20260808 11:44:29.086817 22851 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:33359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab.
Entry for principal oryx/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab.
I20260808 11:44:29.114043 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45849
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab
--principal=oryx/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:29.438375 23108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:29.438674 23108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:29.438835 23108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:29.445822 23108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:29.445916 23108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:29.446125 23108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:29.455864 23108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab
--principal=oryx/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:29.457010 23108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:29.458210 23108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:29.471014 23116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:29.471516 23118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:29.471081 23113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:29.471956 23108 server_base.cc:1034] running on GCE node
I20260808 11:44:29.472514 23108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:29.473461 23108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:29.474663 23108 hybrid_clock.cc:648] HybridClock initialized: now 1786189469474616 us; error 37 us; skew 500 ppm
I20260808 11:44:29.475066 23108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:29 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189469, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:29.478197 23108 init.cc:377] Logged in from keytab as oryx/127.13.158.66@KRBTEST.COM (short username oryx)
I20260808 11:44:29.479650 23108 webserver.cc:460] Webserver started at http://127.13.158.66:43375/ using document root <none> and password file <none>
I20260808 11:44:29.480309 23108 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:29.480480 23108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:29.480759 23108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:29.484071 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "2ff970853d2f47bdb3eecc00a9310ec4"
format_stamp: "Formatted at 2026-08-08 11:44:29 on dist-test-slave-39l8"
server_key: "e1f28da827f504cab5717b0a79f7af3e"
server_key_iv: "ec7632ab21e18e3b33f9213c9a692d3c"
server_key_version: "encryptionkey@0"
I20260808 11:44:29.484844 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "2ff970853d2f47bdb3eecc00a9310ec4"
format_stamp: "Formatted at 2026-08-08 11:44:29 on dist-test-slave-39l8"
server_key: "e1f28da827f504cab5717b0a79f7af3e"
server_key_iv: "ec7632ab21e18e3b33f9213c9a692d3c"
server_key_version: "encryptionkey@0"
I20260808 11:44:29.489996 23108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 11:44:29.493553 23127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:29.495115 23108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:29.495302 23108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "2ff970853d2f47bdb3eecc00a9310ec4"
format_stamp: "Formatted at 2026-08-08 11:44:29 on dist-test-slave-39l8"
server_key: "e1f28da827f504cab5717b0a79f7af3e"
server_key_iv: "ec7632ab21e18e3b33f9213c9a692d3c"
server_key_version: "encryptionkey@0"
I20260808 11:44:29.495492 23108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:29.509532 23108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:29.513638 23108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:29.513932 23108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:29.514900 23108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:29.516083 23108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:29.516186 23108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:29.516341 23108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:29.516413 23108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:29.546816 23108 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:41193
I20260808 11:44:29.546849 23281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:41193 every 8 connection(s)
I20260808 11:44:29.548096 23108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:29.554919 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 23108
I20260808 11:44:29.555106 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:29.555663 13945 external_mini_cluster.cc:1428] Setting key cbd8a7820ddf2ee09f5b512053dd8514
Aug 08 11:44:29 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189469, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:29.568665 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:29.550542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:57301 (local address 127.13.158.126:45849)
0808 11:44:29.550773 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:29.550779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:29.551935 (+ 1156us) server_negotiation.cc:407] Connection header received
0808 11:44:29.553620 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:29.553625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:29.553693 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:29.553782 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:29.555307 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:29.556366 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:29.558056 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:29.558668 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:29.558824 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:29.561969 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:29.561979 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:29.561986 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:29.562038 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:29.565562 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:29.566336 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:29.566343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:29.566347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:29.566481 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:29.566969 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:29.566975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:29.566978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:29.567259 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:29.567461 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:29.568130 (+ 669us) server_negotiation.cc:299] Negotiation successful
0808 11:44:29.568348 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:44:29.569567 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:29.551054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.66:57301)
0808 11:44:29.551782 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:29.551819 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:29.553379 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:29.553977 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:29.553984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:29.554431 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:29.555117 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:29.555137 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:29.556549 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:29.556553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:29.557911 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:29.557922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:29.558894 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:29.558898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:29.559071 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:29.559709 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:29.559730 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:29.561704 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:29.565710 (+ 4006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:29.565716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:29.565727 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:29.566196 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:29.566597 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:29.566602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:29.566606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:29.566853 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:29.567424 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:29.567434 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:29.567754 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 11:44:29.567999 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 11:44:29.568474 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":298,"threads_started":1}
I20260808 11:44:29.571166 23283 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45849
I20260808 11:44:29.571468 23283 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:29.572206 23283 heartbeater.cc:511] Master 127.13.158.126:45849 requested a full tablet report, sending...
I20260808 11:44:29.573763 22851 ts_manager.cc:194] Registered new tserver with Master: 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66:41193)
I20260808 11:44:29.575070 22851 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:57301
WARNING: no policy specified for oryx/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab.
Entry for principal oryx/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab.
I20260808 11:44:29.609135 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45849
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab
--principal=oryx/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:29.935508 23290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:29.935793 23290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:29.935897 23290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:29.943082 23290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:29.943198 23290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:29.943308 23290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:29.952848 23290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab
--principal=oryx/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:29.954206 23290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:29.955536 23290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:29.969002 23300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:29.968966 23303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:29.968923 23299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:29.969781 23290 server_base.cc:1034] running on GCE node
I20260808 11:44:29.970413 23290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:29.971453 23290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:29.972663 23290 hybrid_clock.cc:648] HybridClock initialized: now 1786189469972627 us; error 32 us; skew 500 ppm
I20260808 11:44:29.972939 23290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:29 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189469, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:29.980240 23290 init.cc:377] Logged in from keytab as oryx/127.13.158.67@KRBTEST.COM (short username oryx)
I20260808 11:44:29.981981 23290 webserver.cc:460] Webserver started at http://127.13.158.67:40945/ using document root <none> and password file <none>
I20260808 11:44:29.982847 23290 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:29.983000 23290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:29.983336 23290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:29.986493 23290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "964f9eaed9c843cd8ef86ee7c526a783"
format_stamp: "Formatted at 2026-08-08 11:44:29 on dist-test-slave-39l8"
server_key: "139988b82c9ad00bdc40ca2511dada50"
server_key_iv: "e2277de9e7103fbcbb75bb12b4ed57c8"
server_key_version: "encryptionkey@0"
I20260808 11:44:29.987406 23290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "964f9eaed9c843cd8ef86ee7c526a783"
format_stamp: "Formatted at 2026-08-08 11:44:29 on dist-test-slave-39l8"
server_key: "139988b82c9ad00bdc40ca2511dada50"
server_key_iv: "e2277de9e7103fbcbb75bb12b4ed57c8"
server_key_version: "encryptionkey@0"
I20260808 11:44:29.992707 23290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:44:29.996198 23315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:29.997457 23290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:44:29.997660 23290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "964f9eaed9c843cd8ef86ee7c526a783"
format_stamp: "Formatted at 2026-08-08 11:44:29 on dist-test-slave-39l8"
server_key: "139988b82c9ad00bdc40ca2511dada50"
server_key_iv: "e2277de9e7103fbcbb75bb12b4ed57c8"
server_key_version: "encryptionkey@0"
I20260808 11:44:29.997870 23290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:30.022697 23290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:30.026526 23290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:30.026859 23290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:30.027804 23290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:30.029028 23290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:30.029129 23290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:30.029273 23290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:30.029338 23290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:30.062803 23290 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:40381
I20260808 11:44:30.062829 23469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:40381 every 8 connection(s)
I20260808 11:44:30.064039 23290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:30.070416 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 23290
I20260808 11:44:30.070598 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:30.071112 13945 external_mini_cluster.cc:1428] Setting key 39b3a29206b0fa21f66ae00f3bf0f07a
Aug 08 11:44:30 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189469, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:30.083318 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.066618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:36899 (local address 127.13.158.126:45849)
0808 11:44:30.066857 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:30.066862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:30.067950 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 11:44:30.069732 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:30.069739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:30.069843 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:30.069955 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:30.071645 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.072500 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.074034 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.074428 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.074570 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.077792 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:30.077796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:30.077799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:30.077830 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:30.080278 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.081029 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.081034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.081036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.081161 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.081604 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.081614 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.081616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.081888 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:30.082043 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:30.082740 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 11:44:30.082960 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:44:30.084224 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.067110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.67:36899)
0808 11:44:30.067791 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:30.067831 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:30.069470 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:30.070209 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:30.070216 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:30.070824 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:30.071463 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.071484 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.072658 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.072669 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.073857 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.073869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.074659 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.074669 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.074918 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.075540 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:30.075561 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:30.077600 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:30.080439 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.080448 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.080463 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.080866 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.081294 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.081300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.081303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.081477 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.082025 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:30.082033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:30.082348 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 11:44:30.082602 (+ 254us) client_negotiation.cc:240] Negotiation successful
0808 11:44:30.083140 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":260,"threads_started":1}
I20260808 11:44:30.085677 23470 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45849
I20260808 11:44:30.085987 23470 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:30.086796 23470 heartbeater.cc:511] Master 127.13.158.126:45849 requested a full tablet report, sending...
I20260808 11:44:30.088191 22851 ts_manager.cc:194] Registered new tserver with Master: 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381)
I20260808 11:44:30.089020 22851 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:36899
I20260808 11:44:30.089952 23101 heartbeater.cc:503] Master 127.13.158.126:45849 was elected leader, sending a full tablet report...
I20260808 11:44:30.098407 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.158.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.125.keytab.
Entry for principal oryx/127.13.158.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.125.keytab.
I20260808 11:44:30.149705 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.125:44621
--webserver_interface=127.13.158.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.125.keytab
--principal=oryx/127.13.158.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:45849,127.13.158.125:44621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:30.476253 23476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:30.476522 23476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:30.476609 23476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:30.484272 23476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:30.484375 23476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:30.484418 23476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:30.484475 23476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:30.484519 23476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:30.494251 23476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:45849,127.13.158.125:44621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.125.keytab
--principal=oryx/127.13.158.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.125:44621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.158.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
W20260808 11:44:30.494542 23476 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.158.126:45849,127.13.158.125:44621'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 11:44:30.495502 23476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:30.496811 23476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:30.508749 23482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:30.508862 23489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:30.508724 23484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:30.509509 23476 server_base.cc:1034] running on GCE node
I20260808 11:44:30.510293 23476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:30.511579 23476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:30.512806 23476 hybrid_clock.cc:648] HybridClock initialized: now 1786189470512748 us; error 38 us; skew 500 ppm
I20260808 11:44:30.513116 23476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:30 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189470, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:30.516129 23476 init.cc:377] Logged in from keytab as oryx/127.13.158.125@KRBTEST.COM (short username oryx)
I20260808 11:44:30.517441 23476 webserver.cc:460] Webserver started at http://127.13.158.125:34475/ using document root <none> and password file <none>
I20260808 11:44:30.518052 23476 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:30.518189 23476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:30.518497 23476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:30.521397 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data/instance:
uuid: "e40f3063557944ce8a011945c0f3c76f"
format_stamp: "Formatted at 2026-08-08 11:44:30 on dist-test-slave-39l8"
server_key: "1964fc69ae665a9f0fce273db7affc43"
server_key_iv: "204a6c04887dc08651168bc4244e92ea"
server_key_version: "encryptionkey@0"
I20260808 11:44:30.522151 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal/instance:
uuid: "e40f3063557944ce8a011945c0f3c76f"
format_stamp: "Formatted at 2026-08-08 11:44:30 on dist-test-slave-39l8"
server_key: "1964fc69ae665a9f0fce273db7affc43"
server_key_iv: "204a6c04887dc08651168bc4244e92ea"
server_key_version: "encryptionkey@0"
I20260808 11:44:30.527024 23476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260808 11:44:30.530364 23499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:30.531615 23476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:30.531788 23476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
uuid: "e40f3063557944ce8a011945c0f3c76f"
format_stamp: "Formatted at 2026-08-08 11:44:30 on dist-test-slave-39l8"
server_key: "1964fc69ae665a9f0fce273db7affc43"
server_key_iv: "204a6c04887dc08651168bc4244e92ea"
server_key_version: "encryptionkey@0"
I20260808 11:44:30.531996 23476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:30.570045 23476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:30.573984 23476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:30.574257 23476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:30.577404 23283 heartbeater.cc:503] Master 127.13.158.126:45849 was elected leader, sending a full tablet report...
I20260808 11:44:30.593816 23476 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.125:44621
I20260808 11:44:30.593837 23565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.125:44621 every 8 connection(s)
I20260808 11:44:30.595290 23476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
I20260808 11:44:30.598937 23566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:30.601233 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 23476
I20260808 11:44:30.606496 23566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:44:30 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.125@KRBTEST.COM
Aug 08 11:44:30 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189470, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.125@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:30.625129 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.603584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58066 (local address 127.13.158.125:44621)
0808 11:44:30.604468 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:30.604479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:30.604526 (+ 47us) server_negotiation.cc:407] Connection header received
0808 11:44:30.606082 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:30.606105 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:30.606452 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:30.606825 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:30.608696 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.610908 (+ 2212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.612331 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.612966 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.613104 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.617490 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:30.617501 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:30.617514 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:30.617555 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:30.620729 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.621666 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.621676 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.621684 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.621866 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.622276 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.622281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.622284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.622964 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:30.623192 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:30.623733 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 11:44:30.624150 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":408,"threads_started":1}
I20260808 11:44:30.625597 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.609502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.13.158.126:45849)
0808 11:44:30.609948 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:30.609953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:30.610773 (+ 820us) server_negotiation.cc:407] Connection header received
0808 11:44:30.610987 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:30.610992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:30.611094 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:30.611230 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:30.612394 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.613192 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.614651 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.615399 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.615543 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.618943 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:30.618947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:30.618951 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:30.618982 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:30.622573 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.623410 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.623413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.623415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.623595 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.624063 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.624067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.624069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.624371 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:30.624532 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:30.625021 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 11:44:30.625238 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":258,"threads_started":1}
I20260808 11:44:30.625665 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.609630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:34206)
0808 11:44:30.610617 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:30.610661 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:30.610853 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:30.611284 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:30.611289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:30.611906 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:30.612206 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.612215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.613323 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.613326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.614494 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.614516 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.615631 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.615636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.615753 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.616398 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:30.616431 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:30.618691 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:30.622701 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.622751 (+ 50us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.622758 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.623291 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.623729 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.623735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.623739 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.623922 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.624510 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:30.624518 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:30.624699 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 11:44:30.624903 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 11:44:30.625206 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"thread_start_us":525,"threads_started":1}
I20260808 11:44:30.628191 23566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:44:30 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189470, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.125@KRBTEST.COM for oryx/127.13.158.125@KRBTEST.COM
I20260808 11:44:30.640166 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.628786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.0.0.1:58076)
0808 11:44:30.628939 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:30.628954 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:30.629057 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:30.629337 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:30.629341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:30.629678 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:30.629796 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.629803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.630748 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.630758 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.631542 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.631552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.632265 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.632271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.632521 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.633085 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:30.633101 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:30.635100 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:30.637852 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.637858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.637862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.638201 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.638549 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.638555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.638561 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.638695 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.639196 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:30.639200 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:30.639331 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 11:44:30.639467 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 11:44:30.639776 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":83}
I20260808 11:44:30.640232 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.628962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.13.158.125:44621)
0808 11:44:30.629101 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:30.629116 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:30.629132 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:44:30.629171 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:30.629175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:30.629231 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:30.629323 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:30.629924 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.630585 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.631684 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.632124 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.632259 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.635266 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:30.635272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:30.635276 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:30.635311 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:30.637708 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.638312 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.638319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.638322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.638445 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.638822 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.638827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.638830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.639072 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:30.639214 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:30.639562 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 11:44:30.639895 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":155,"server-negotiator.queue_time_us":47}
I20260808 11:44:30.644865 23566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Bootstrap starting.
I20260808 11:44:30.647907 23566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:30.648864 23566 log.cc:824] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:30.651340 23566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: No bootstrap required, opened a new log
I20260808 11:44:30.657905 23566 raft_consensus.cc:348] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } }
I20260808 11:44:30.658200 23566 raft_consensus.cc:374] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:30.658260 23566 raft_consensus.cc:729] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40f3063557944ce8a011945c0f3c76f, State: Initialized, Role: FOLLOWER
I20260808 11:44:30.658775 23566 consensus_queue.cc:260] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } }
I20260808 11:44:30.659492 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } } }
I20260808 11:44:30.659755 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:44:30.660391 23566 sys_catalog.cc:564] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:44:30.669006 23591 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:44:30.669142 23591 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:44:30.671669 23476 master_runner.cc:428] Detected that this master e40f3063557944ce8a011945c0f3c76f is joining an existing cluster
I20260808 11:44:30.671767 23476 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.158.125:44621
I20260808 11:44:30.707597 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.676523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58084 (local address 127.13.158.125:44621)
0808 11:44:30.676711 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:30.676718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:30.691540 (+ 14822us) server_negotiation.cc:407] Connection header received
0808 11:44:30.691744 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:30.691749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:30.691824 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:30.691910 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:30.693779 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.694948 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.697167 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.697884 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.697978 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.700676 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:30.700682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:30.700690 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:30.700724 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:30.704476 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.705287 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.705293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.705296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.705424 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.705859 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.705864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.705867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.706261 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:30.706408 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:30.707000 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 11:44:30.707239 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:44:30.708678 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.675914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.0.0.1:58084)
0808 11:44:30.691442 (+ 15528us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:30.691460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:30.691624 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:30.692220 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:30.692227 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:30.693194 (+ 967us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:30.693631 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.693643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.695200 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.695205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.696988 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.697003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.698311 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.698322 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.698486 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.699248 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:30.699273 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:30.700477 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:30.704611 (+ 4134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.704616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.704620 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.705164 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.705562 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.705572 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.705576 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.705743 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.706495 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:30.706499 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:30.706669 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 11:44:30.707878 (+ 1209us) client_negotiation.cc:240] Negotiation successful
0808 11:44:30.708202 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15320,"mutex_wait_us":169,"spinlock_wait_cycles":18816,"thread_start_us":15230,"threads_started":1}
I20260808 11:44:30.709431 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.676088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.13.158.126:45849)
0808 11:44:30.676282 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:30.676288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:30.691356 (+ 15068us) server_negotiation.cc:407] Connection header received
0808 11:44:30.691646 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:30.691653 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:30.691755 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:30.691834 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:30.693547 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.694903 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.697415 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.698203 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.698315 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.701330 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:30.701341 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:30.701346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:30.701389 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:30.705639 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.706539 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.706547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.706553 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.706741 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.707242 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.707249 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.707255 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.707638 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:30.707912 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:30.708565 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 11:44:30.708972 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for oryx/127.13.158.124@KRBTEST.COM; defaulting to no policy
I20260808 11:44:30.710639 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.675893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:34212)
0808 11:44:30.691241 (+ 15348us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:30.691264 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:30.691480 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:30.692098 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:30.692105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:30.692903 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:30.693329 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.693366 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.695079 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.695087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.697236 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.697256 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.698744 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.698754 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.698972 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.699618 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:30.699700 (+ 82us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:30.701055 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:30.705826 (+ 4771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.705834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.705840 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.706383 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.706929 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.706935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.706940 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.707111 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.708009 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:30.708014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:30.708230 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 11:44:30.709716 (+ 1486us) client_negotiation.cc:240] Negotiation successful
0808 11:44:30.710128 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15177,"mutex_wait_us":129,"thread_start_us":15006,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.124@KRBTEST.COM" created.
Aug 08 11:44:30 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189468, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.126@KRBTEST.COM for oryx/127.13.158.125@KRBTEST.COM
I20260808 11:44:30.731040 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.716688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58094 (local address 127.13.158.125:44621)
0808 11:44:30.716959 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:30.716965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:30.717454 (+ 489us) server_negotiation.cc:407] Connection header received
0808 11:44:30.717777 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:30.717783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:30.717910 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:30.718043 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:30.719560 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.720641 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.721972 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.722447 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.722588 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.725663 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:30.725668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:30.725671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:30.725697 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:30.728283 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.729034 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.729040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.729044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.729184 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.729564 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.729568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.729574 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.729814 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:30.729966 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:30.730449 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 11:44:30.730658 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260808 11:44:30.731094 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.716650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.0.0.1:58094)
0808 11:44:30.717328 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:30.717369 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:30.717589 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:30.718111 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:30.718121 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:30.719068 (+ 947us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:30.719395 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.719411 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.720798 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.720806 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.721802 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.721814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.722619 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.722625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.722840 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.723432 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:30.723448 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:30.725483 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:30.728468 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.728481 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.728490 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.728915 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.729308 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.729314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.729317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.729450 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.729961 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:30.729970 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:30.730133 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 11:44:30.730337 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 11:44:30.730657 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":315,"threads_started":1}
I20260808 11:44:30.733263 22851 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.158.125:44621: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e40f3063557944ce8a011945c0f3c76f"
member_type: NON_VOTER
last_known_addr {
host: "127.13.158.125"
port: 44621
}
attrs {
promote: true
}
}
dest_uuid: "560789189ca9482d8310a358e6abfd54"
cas_config_opid_index: -1
I20260808 11:44:30.734192 22851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } }
I20260808 11:44:30.739193 23612 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e40f3063557944ce8a011945c0f3c76f (127.13.158.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } }
I20260808 11:44:30.740274 23533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:30.744220 23533 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Refusing update from remote peer 560789189ca9482d8310a358e6abfd54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 11:44:30.744298 23612 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.158.125:44621
I20260808 11:44:30.745047 23476 master.cc:537] Master@127.13.158.125:44621 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.124.keytab.
Entry for principal oryx/127.13.158.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.124.keytab.
I20260808 11:44:30.747880 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.124:39083
--webserver_interface=127.13.158.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.124.keytab
--principal=oryx/127.13.158.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:45849,127.13.158.125:44621,127.13.158.124:39083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:44:30.758879 23611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } }
I20260808 11:44:30.759582 23611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:30.760166 23612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:30.760607 23476 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:30.761098 23476 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:30.761399 23476 tablet_replica.cc:330] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: stopping tablet replica
W20260808 11:44:30.767458 22831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from unknown peer (error 108)
I20260808 11:44:30.768445 23476 master.cc:552] Master@127.13.158.125:44621 shutdown complete.
I20260808 11:44:30.768586 23476 master_runner.cc:305] Clearing existing system tablet
I20260808 11:44:30.769050 23476 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:30.769212 23476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:30.783113 23476 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.010s sys 0.002s
I20260808 11:44:30.783159 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.782377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.0.0.1:58096)
0808 11:44:30.782615 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:30.782881 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 11:44:30.783589 22831 consensus_peers.cc:487] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 -> Peer e40f3063557944ce8a011945c0f3c76f (127.13.158.125:44621): Couldn't send request to peer e40f3063557944ce8a011945c0f3c76f. Status: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44621: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 11:44:30.785197 23620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:30.785554 23476 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:30.785887 23476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 11:44:30.785992 23476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
uuid: "e40f3063557944ce8a011945c0f3c76f"
format_stamp: "Formatted at 2026-08-08 11:44:30 on dist-test-slave-39l8"
server_key: "1964fc69ae665a9f0fce273db7affc43"
server_key_iv: "204a6c04887dc08651168bc4244e92ea"
server_key_version: "encryptionkey@0"
I20260808 11:44:30.791764 23476 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:30.798869 23476 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 11:44:30.799064 23476 log.cc:1196] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 11:44:30.799494 23476 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Deleting consensus metadata
I20260808 11:44:30.799757 23476 master_runner.cc:315] Copying system tablet from 127.13.158.126:45849
I20260808 11:44:30.808318 23476 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: tablet copy: Beginning tablet copy session from remote peer at address 127.13.158.126:45849
I20260808 11:44:30.830124 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.809343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.13.158.126:45849)
0808 11:44:30.809542 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:30.809548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:30.815340 (+ 5792us) server_negotiation.cc:407] Connection header received
0808 11:44:30.815557 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:30.815564 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:30.815675 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:30.815783 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:30.816992 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.818210 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.820103 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.820748 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.820890 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.823298 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:30.823304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:30.823309 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:30.823350 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:30.826891 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.827793 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.827798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.827803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.827941 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:30.828391 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:30.828399 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:30.828403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:30.828811 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:30.828991 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:30.829512 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 11:44:30.829754 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 11:44:30.830147 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:30.809095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:34224)
0808 11:44:30.815191 (+ 6096us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:30.815213 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:30.815390 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:30.815880 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:30.815885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:30.816461 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:30.816810 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.816832 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.818386 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.818392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.819942 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:30.819957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:30.820948 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:30.820956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:30.821142 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:30.821899 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:30.821924 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:30.823072 (+ 1148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:30.827055 (+ 3983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.827061 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.827067 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.827648 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.828077 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:30.828083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:30.828086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:30.828243 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:30.828967 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:30.828972 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:30.829155 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 11:44:30.829330 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 11:44:30.829682 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5945,"thread_start_us":5805,"threads_started":1}
I20260808 11:44:30.831619 22876 tablet_copy_service.cc:140] P 560789189ca9482d8310a358e6abfd54: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e40f3063557944ce8a011945c0f3c76f ({username='oryx', principal='oryx/127.13.158.125@KRBTEST.COM'} at 127.0.0.1:34224)
I20260808 11:44:30.831948 22876 tablet_copy_service.cc:161] P 560789189ca9482d8310a358e6abfd54: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e40f3063557944ce8a011945c0f3c76f at {username='oryx', principal='oryx/127.13.158.125@KRBTEST.COM'} at 127.0.0.1:34224: session id = e40f3063557944ce8a011945c0f3c76f-00000000000000000000000000000000
I20260808 11:44:30.836385 22876 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 11:44:30.838690 23476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 11:44:30.845405 23476 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: tablet copy: Starting download of 0 data blocks...
I20260808 11:44:30.845738 23476 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: tablet copy: Starting download of 1 WAL segments...
I20260808 11:44:30.848445 23476 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 11:44:30.852743 22876 tablet_copy_service.cc:342] P 560789189ca9482d8310a358e6abfd54: Request end of tablet copy session e40f3063557944ce8a011945c0f3c76f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.158.125@KRBTEST.COM'} at 127.0.0.1:34224
I20260808 11:44:30.852952 22876 tablet_copy_service.cc:433] P 560789189ca9482d8310a358e6abfd54: ending tablet copy session e40f3063557944ce8a011945c0f3c76f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e40f3063557944ce8a011945c0f3c76f
I20260808 11:44:30.864882 23476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:30.867583 23634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:30.872017 23637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:30.880239 23476 server_base.cc:1034] running on GCE node
W20260808 11:44:30.883898 23640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:30.884454 23476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:30.885291 23476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:30.886528 23476 hybrid_clock.cc:648] HybridClock initialized: now 1786189470886466 us; error 34 us; skew 500 ppm
I20260808 11:44:30.886863 23476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:30 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189470, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:30.889170 23476 init.cc:377] Logged in from keytab as oryx/127.13.158.125@KRBTEST.COM (short username oryx)
I20260808 11:44:30.890460 23476 webserver.cc:460] Webserver started at http://127.13.158.125:36379/ using document root <none> and password file <none>
I20260808 11:44:30.890861 23476 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:30.890941 23476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:30.894178 23476 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 11:44:30.896203 23648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:30.896674 23476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 11:44:30.896814 23476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
uuid: "e40f3063557944ce8a011945c0f3c76f"
format_stamp: "Formatted at 2026-08-08 11:44:30 on dist-test-slave-39l8"
server_key: "1964fc69ae665a9f0fce273db7affc43"
server_key_iv: "204a6c04887dc08651168bc4244e92ea"
server_key_version: "encryptionkey@0"
I20260808 11:44:30.896978 23476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:30.923377 23476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:30.927481 23476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:30.954809 23476 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.125:44621
I20260808 11:44:30.954877 23715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.125:44621 every 8 connection(s)
I20260808 11:44:30.955791 23476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
I20260808 11:44:30.962504 23716 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:44:30.966418 23716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Bootstrap starting.
I20260808 11:44:30.984030 23716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:44:30.984618 23716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: Bootstrap complete.
I20260808 11:44:30.985787 23716 raft_consensus.cc:348] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } }
I20260808 11:44:30.986085 23716 raft_consensus.cc:729] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e40f3063557944ce8a011945c0f3c76f, State: Initialized, Role: LEARNER
I20260808 11:44:30.986342 23716 consensus_queue.cc:260] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } }
I20260808 11:44:30.987457 23716 sys_catalog.cc:564] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:30.987180 23722 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } }
I20260808 11:44:30.987733 23722 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: This master's current role is: LEARNER
I20260808 11:44:31.001492 23739 catalog_manager.cc:1259] Loaded cluster ID: d7473c94efbd4a0085f2b819a32e48a1
I20260808 11:44:31.001611 23739 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: loading cluster ID for follower catalog manager: success
I20260808 11:44:31.005239 23739 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: acquiring CA information for follower catalog manager: success
I20260808 11:44:31.007694 23739 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:44:31.091984 23470 heartbeater.cc:503] Master 127.13.158.126:45849 was elected leader, sending a full tablet report...
W20260808 11:44:31.256353 23615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:31.256906 23615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:31.257186 23615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 11:44:31.263332 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.255050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.0.0.1:58112)
0808 11:44:31.255276 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.255298 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.255458 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.256890 (+ 1432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.256897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.256934 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:31.257287 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.257304 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.259089 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.259094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.260579 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.260591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.262191 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.262206 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.262501 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.262574 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.262694 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.262912 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 11:44:31.263419 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.255271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58112 (local address 127.13.158.125:44621)
0808 11:44:31.256429 (+ 1158us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.256437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.256484 (+ 47us) server_negotiation.cc:407] Connection header received
0808 11:44:31.256572 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.256577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.256669 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.256807 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:31.257457 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.258916 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.260767 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.261952 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.262237 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.262497 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.262743 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.262936 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":680,"threads_started":1}
I20260808 11:44:31.264801 23687 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 11:44:31.265475 23722 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 560789189ca9482d8310a358e6abfd54. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } }
I20260808 11:44:31.265748 23722 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: This master's current role is: LEARNER
W20260808 11:44:31.267925 23615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:31.268106 23615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:31.268236 23615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:31.268406 23615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:31.268518 23615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:31.280831 23615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:45849,127.13.158.125:44621,127.13.158.124:39083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.124.keytab
--principal=oryx/127.13.158.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.124:39083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.158.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:31.282662 23615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:31.284296 23615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:31.300813 23752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:31.300904 23755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:31.301304 23750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:31.302896 23615 server_base.cc:1034] running on GCE node
I20260808 11:44:31.303766 23615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:31.305428 23615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:31.306752 23615 hybrid_clock.cc:648] HybridClock initialized: now 1786189471306664 us; error 47 us; skew 500 ppm
I20260808 11:44:31.307128 23615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:31 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189471, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:31.311453 23615 init.cc:377] Logged in from keytab as oryx/127.13.158.124@KRBTEST.COM (short username oryx)
I20260808 11:44:31.313462 23615 webserver.cc:460] Webserver started at http://127.13.158.124:41769/ using document root <none> and password file <none>
I20260808 11:44:31.314337 23615 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:31.314528 23615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:31.314960 23615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:31.318656 23615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data/instance:
uuid: "365c520c6f164df1a8cf66562b8bea8c"
format_stamp: "Formatted at 2026-08-08 11:44:31 on dist-test-slave-39l8"
server_key: "dc92d46bb0bdfb0c20dd1b6cb7487456"
server_key_iv: "f3d34772b2c72f2730b8870bc5bfeaa2"
server_key_version: "encryptionkey@0"
I20260808 11:44:31.319849 23615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal/instance:
uuid: "365c520c6f164df1a8cf66562b8bea8c"
format_stamp: "Formatted at 2026-08-08 11:44:31 on dist-test-slave-39l8"
server_key: "dc92d46bb0bdfb0c20dd1b6cb7487456"
server_key_iv: "f3d34772b2c72f2730b8870bc5bfeaa2"
server_key_version: "encryptionkey@0"
I20260808 11:44:31.325966 23615 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260808 11:44:31.330646 23769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:31.332191 23615 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 11:44:31.332386 23615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
uuid: "365c520c6f164df1a8cf66562b8bea8c"
format_stamp: "Formatted at 2026-08-08 11:44:31 on dist-test-slave-39l8"
server_key: "dc92d46bb0bdfb0c20dd1b6cb7487456"
server_key_iv: "f3d34772b2c72f2730b8870bc5bfeaa2"
server_key_version: "encryptionkey@0"
I20260808 11:44:31.332599 23615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:31.365552 23615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:31.369879 23615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:31.370110 23615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:31.390170 23615 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.124:39083
I20260808 11:44:31.390230 23835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.124:39083 every 8 connection(s)
I20260808 11:44:31.391572 23615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
I20260808 11:44:31.395668 23836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:31.399230 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 23615
I20260808 11:44:31.403714 23836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:44:31 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.124@KRBTEST.COM
Aug 08 11:44:31 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189471, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.124@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:31.421837 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.407220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.13.158.126:45849)
0808 11:44:31.407748 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.407755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.408254 (+ 499us) server_negotiation.cc:407] Connection header received
0808 11:44:31.408384 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.408387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.408455 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.408557 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.409799 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.410857 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.412496 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.412978 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.413113 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.416524 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.416528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.416531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.416562 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.419040 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.419718 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.419726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.419731 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.419857 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.420276 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.420282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.420285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.420531 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.420680 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.421231 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.421431 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":251,"threads_started":1}
I20260808 11:44:31.422777 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.407202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54254)
0808 11:44:31.408072 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.408130 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.408278 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.408638 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.408645 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.409387 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:31.409664 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.409673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.411011 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.411016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.412345 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.412367 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.413206 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.413213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.413377 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.413960 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:31.413981 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:31.416338 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:31.419163 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.419176 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.419187 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.419595 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.419969 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.419974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.419978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.420144 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.420657 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:31.420665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:31.420889 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.421136 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.421575 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":2,"thread_start_us":507,"threads_started":1}
I20260808 11:44:31.422797 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.401696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.13.158.124:39083)
0808 11:44:31.402648 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.402660 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.402802 (+ 142us) server_negotiation.cc:407] Connection header received
0808 11:44:31.404435 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.404456 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.404815 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.405135 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.406951 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.408979 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.410551 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.411411 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.411589 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.416087 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.416095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.416106 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.416141 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.419159 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.419861 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.419869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.419872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.420025 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.420417 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.420422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.420427 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.420872 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.421199 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.421511 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.421750 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":527,"threads_started":1}
I20260808 11:44:31.424360 23836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:44:31 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189471, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.124@KRBTEST.COM for oryx/127.13.158.125@KRBTEST.COM
I20260808 11:44:31.437546 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.425120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.0.0.1:40914)
0808 11:44:31.425312 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.425332 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.425426 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.425740 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.425745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.426066 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:31.426185 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.426192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.427198 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.427209 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.428085 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.428095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.428806 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.428811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.429135 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.429615 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:31.429631 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:31.432055 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:31.435114 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.435119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.435122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.435534 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.435881 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.435884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.435886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.436049 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.436668 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:31.436671 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:31.436868 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.436984 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.437229 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 11:44:31.437800 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.425227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.13.158.125:44621)
0808 11:44:31.425463 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.425471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.425491 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:44:31.425553 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.425557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.425613 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.425714 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.426337 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.427064 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.428240 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.428636 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.428768 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.432229 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.432234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.432237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.432269 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.435000 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.435626 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.435629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.435631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.435789 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.436138 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.436141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.436143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.436574 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.436681 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.437100 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.437365 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260808 11:44:31.438827 23836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:44:31 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189471, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.124@KRBTEST.COM for oryx/127.13.158.124@KRBTEST.COM
I20260808 11:44:31.455509 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.439722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.0.0.1:43640)
0808 11:44:31.439832 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.439846 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.439977 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.440293 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.440297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.440794 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:31.440976 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.441001 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.442380 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.442383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.443468 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.443479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.444643 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.444647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.444940 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.445589 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:31.445605 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:31.447916 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:31.452672 (+ 4756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.452675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.452678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.453098 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.453546 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.453549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.453551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.453763 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.454475 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:31.454478 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:31.454605 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.454818 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.455007 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":31}
I20260808 11:44:31.455700 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.439525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.13.158.124:39083)
0808 11:44:31.439685 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.439689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.439923 (+ 234us) server_negotiation.cc:407] Connection header received
0808 11:44:31.440097 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.440133 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.440181 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.440276 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.441144 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.442254 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.443626 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.444491 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.444635 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.448151 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.448170 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.448174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.448221 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.452528 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.453227 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.453232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.453236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.453438 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.453872 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.453877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.453881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.454366 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.454503 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.454938 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.455196 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":62}
I20260808 11:44:31.463450 23836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Bootstrap starting.
I20260808 11:44:31.467563 23836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:31.469071 23836 log.cc:824] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:31.472239 23836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: No bootstrap required, opened a new log
I20260808 11:44:31.482282 23836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } }
I20260808 11:44:31.482748 23836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:31.482971 23836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365c520c6f164df1a8cf66562b8bea8c, State: Initialized, Role: FOLLOWER
I20260808 11:44:31.483575 23836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } }
I20260808 11:44:31.484719 23847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } } }
I20260808 11:44:31.485316 23847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:44:31.485682 23836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:44:31.500324 23866 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:44:31.500455 23866 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:44:31.505260 23615 master_runner.cc:428] Detected that this master 365c520c6f164df1a8cf66562b8bea8c is joining an existing cluster
I20260808 11:44:31.505440 23615 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.158.124:39083
I20260808 11:44:31.529593 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 22934
I20260808 11:44:31.543393 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.511147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54256 (local address 127.13.158.126:45849)
0808 11:44:31.511329 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.511336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.514502 (+ 3166us) server_negotiation.cc:407] Connection header received
0808 11:44:31.514895 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.514904 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.515012 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.515101 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.517955 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.519728 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.523159 (+ 3431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.524308 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.524436 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.532895 (+ 8459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.532902 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.532907 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.532950 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.538409 (+ 5459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.539400 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.539405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.539409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.539665 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.540553 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.540559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.540562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.541061 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.541233 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.542692 (+ 1459us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.543034 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:44:31.546015 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.511975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54256)
0808 11:44:31.514364 (+ 2389us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.514406 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.514670 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.515363 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.515370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.516894 (+ 1524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:31.517730 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.517749 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.519976 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.519984 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.522944 (+ 2960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.522965 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.524852 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.524862 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.525183 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.526572 (+ 1389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:31.526605 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:31.532627 (+ 6022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:31.538583 (+ 5956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.538589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.538593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.539214 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.539879 (+ 665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.539884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.539888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.540326 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.541340 (+ 1014us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:31.541345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:31.541712 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.541906 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.542346 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2144,"thread_start_us":14880,"threads_started":1}
Aug 08 11:44:31 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.65@KRBTEST.COM
I20260808 11:44:31.570906 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.511970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40920 (local address 127.13.158.125:44621)
0808 11:44:31.512210 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.512217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.528621 (+ 16404us) server_negotiation.cc:407] Connection header received
0808 11:44:31.528898 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.528904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.528988 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.529076 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.535240 (+ 6164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.537052 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.540881 (+ 3829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.542064 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.542199 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.552200 (+ 10001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.552219 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.552224 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.552268 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.559848 (+ 7580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.564217 (+ 4369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.564224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.564229 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.564429 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.565010 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.565015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.565030 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.565489 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.565662 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.569811 (+ 4149us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.570219 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 11:44:31.572383 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.514457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.0.0.1:40920)
0808 11:44:31.528512 (+ 14055us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.528535 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.528734 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.530174 (+ 1440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.530184 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.532454 (+ 2270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:31.535005 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.535022 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.537372 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.537380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.540601 (+ 3221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.540622 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.542677 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.542686 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.543082 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.547772 (+ 4690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:31.547794 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:31.549535 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:31.560059 (+ 10524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.560064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.560068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.563251 (+ 3183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.564596 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.564599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.564602 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.564848 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.565749 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:31.565754 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:31.567902 (+ 2148us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.571201 (+ 3299us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.571586 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13808,"mutex_wait_us":3,"thread_start_us":12809,"threads_started":1}
I20260808 11:44:31.573539 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.511378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.13.158.124:39083)
0808 11:44:31.511629 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.511636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.527668 (+ 16032us) server_negotiation.cc:407] Connection header received
0808 11:44:31.527847 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.527853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.527946 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.528058 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.532261 (+ 4203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.533983 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.545304 (+ 11321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.546437 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.546661 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.553209 (+ 6548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.553215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.553220 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.553255 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.562606 (+ 9351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.566638 (+ 4032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.566652 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.566659 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.567366 (+ 707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.568139 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.568146 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.568152 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.568826 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.569039 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.572743 (+ 3704us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.573012 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":50,"spinlock_wait_cycles":956800}
I20260808 11:44:31.580935 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.530871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47850 (local address 127.13.158.65:38839)
0808 11:44:31.547282 (+ 16411us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.547292 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.547345 (+ 53us) server_negotiation.cc:407] Connection header received
0808 11:44:31.547414 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.547423 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.547854 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.550973 (+ 3119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.551371 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.553651 (+ 2280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.556529 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.557808 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.566059 (+ 8251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.566674 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.566690 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.566735 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.566791 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.575116 (+ 8325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.576340 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.576349 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.576353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.576563 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.577149 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.577154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.577157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.577808 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.579692 (+ 1884us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.579881 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.580270 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16179,"thread_start_us":17172,"threads_started":1}
I20260808 11:44:31.582134 23876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.512271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.0.0.1:43652)
0808 11:44:31.527555 (+ 15284us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.527570 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.527720 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.528254 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.528258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.530055 (+ 1797us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:31.530971 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.530985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.543801 (+ 12816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.543806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.545150 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.545162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.547171 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.547181 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.547378 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.549865 (+ 2487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:31.549886 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:31.551867 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:31.563409 (+ 11542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.563416 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.563421 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.564068 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.567520 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.567525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.567528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.567749 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.569137 (+ 1388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:31.569141 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:31.569393 (+ 252us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.581047 (+ 11654us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.581395 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15175,"thread_start_us":1240,"threads_started":1}
I20260808 11:44:31.583274 23091 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 08 11:44:31 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189468, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.126@KRBTEST.COM for oryx/127.13.158.124@KRBTEST.COM
I20260808 11:44:31.596210 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.558333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.0.0.1:43656)
0808 11:44:31.558593 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.558617 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.558755 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.574253 (+ 15498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.574259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.575021 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:31.575365 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.575380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.577259 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.577265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.579135 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.579151 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.582649 (+ 3498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.582657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.582931 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.583747 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:31.583774 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:31.587077 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:31.592503 (+ 5426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.592512 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.592519 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.593252 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.593782 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.593788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.593791 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.594160 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.594987 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:31.594994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:31.595247 (+ 253us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.595410 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.595705 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 11:44:31.596379 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.558863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.13.158.124:39083)
0808 11:44:31.573771 (+ 14908us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.573776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.573793 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:44:31.573885 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.573890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.574045 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.575561 (+ 1516us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.575793 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.577082 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.579835 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.580916 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.583307 (+ 2391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.587301 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.587307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.587312 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.587344 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.592335 (+ 4991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.593397 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.593405 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.593410 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.593632 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.594312 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.594320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.594324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.594760 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.594915 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.595575 (+ 660us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.595827 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14814,"thread_start_us":29887,"threads_started":1}
I20260808 11:44:31.598474 22850 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.158.124:39083: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c"
member_type: NON_VOTER
last_known_addr {
host: "127.13.158.124"
port: 39083
}
attrs {
promote: true
}
}
dest_uuid: "560789189ca9482d8310a358e6abfd54"
cas_config_opid_index: 5
I20260808 11:44:31.600548 22850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } }
I20260808 11:44:31.603773 23612 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 365c520c6f164df1a8cf66562b8bea8c (127.13.158.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.604794 23804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:31.607590 23612 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.158.124:39083
I20260808 11:44:31.608803 23687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 LEARNER]: Refusing update from remote peer 560789189ca9482d8310a358e6abfd54: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 11:44:31.608518 23882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.609635 23882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:31.609623 23612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 11:44:31.619529 23687 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 365c520c6f164df1a8cf66562b8bea8c (127.13.158.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.626009 23722 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: NON_VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: true } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.626443 23722 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: This master's current role is: LEARNER
I20260808 11:44:31.632194 23882 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: attempting to promote NON_VOTER e40f3063557944ce8a011945c0f3c76f to VOTER
I20260808 11:44:31.633345 23882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } }
W20260808 11:44:31.635332 23776 tablet.cc:2369] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:44:31.636749 23804 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Refusing update from remote peer 560789189ca9482d8310a358e6abfd54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 11:44:31.637476 23615 master.cc:537] Master@127.13.158.124:39083 shutting down...
I20260808 11:44:31.639529 23802 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Refusing update from remote peer 560789189ca9482d8310a358e6abfd54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 11:44:31.643083 23612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:31.641120 23687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 LEARNER]: Refusing update from remote peer 560789189ca9482d8310a358e6abfd54: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 11:44:31.653707 23615 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:31.653837 23612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 11:44:31.654165 23615 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:31.654389 23615 tablet_replica.cc:330] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: stopping tablet replica
I20260808 11:44:31.656733 23612 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e40f3063557944ce8a011945c0f3c76f (127.13.158.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.659349 23685 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e40f3063557944ce8a011945c0f3c76f (127.13.158.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.661890 23615 master.cc:552] Master@127.13.158.124:39083 shutdown complete.
I20260808 11:44:31.662024 23615 master_runner.cc:305] Clearing existing system tablet
I20260808 11:44:31.662431 23615 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:31.662588 23615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:31.663331 23722 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.663554 23722 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:44:31.666354 23886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.666765 23886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:31.667984 23615 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.001s
I20260808 11:44:31.669363 23886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.669603 23886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:31.670931 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.670446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.0.0.1:43672)
0808 11:44:31.670617 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.670746 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:39083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 11:44:31.671270 22829 consensus_peers.cc:487] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 -> Peer 365c520c6f164df1a8cf66562b8bea8c (127.13.158.124:39083): Couldn't send request to peer 365c520c6f164df1a8cf66562b8bea8c. Status: Network error: Client connection negotiation failed: client connection to 127.13.158.124:39083: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 11:44:31.675151 23889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:31.675550 23615 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:31.675843 23615 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.003s
I20260808 11:44:31.675948 23615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
uuid: "365c520c6f164df1a8cf66562b8bea8c"
format_stamp: "Formatted at 2026-08-08 11:44:31 on dist-test-slave-39l8"
server_key: "dc92d46bb0bdfb0c20dd1b6cb7487456"
server_key_iv: "f3d34772b2c72f2730b8870bc5bfeaa2"
server_key_version: "encryptionkey@0"
I20260808 11:44:31.681008 23615 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:31.689455 23615 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 11:44:31.689672 23615 log.cc:1196] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 11:44:31.690348 23615 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Deleting consensus metadata
I20260808 11:44:31.690721 23615 master_runner.cc:315] Copying system tablet from 127.13.158.126:45849
I20260808 11:44:31.694092 23615 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: tablet copy: Beginning tablet copy session from remote peer at address 127.13.158.126:45849
I20260808 11:44:31.717460 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.694936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54272)
0808 11:44:31.699186 (+ 4250us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:31.699216 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:31.699374 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:31.699858 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:31.699863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:31.700517 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:31.701051 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.701066 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.703016 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.703021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.704696 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:31.704713 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.706057 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.706063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:31.706275 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.707191 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:31.707214 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:31.708490 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:31.713963 (+ 5473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.713968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.713972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.714534 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.715067 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:31.715072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:31.715075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:31.715307 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:31.716099 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:31.716104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:31.716322 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 11:44:31.716482 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 11:44:31.716846 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3988,"thread_start_us":3900,"threads_started":1}
I20260808 11:44:31.717466 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:31.695253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.13.158.126:45849)
0808 11:44:31.695533 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:31.695542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:31.699374 (+ 3832us) server_negotiation.cc:407] Connection header received
0808 11:44:31.699546 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:31.699552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:31.699694 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:31.699806 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:31.701215 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.702840 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.704868 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:31.705870 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:31.705997 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:31.708758 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:31.708766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:31.708771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:31.708836 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:31.713818 (+ 4982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.714679 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.714685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.714689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.714941 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:31.715426 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:31.715432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:31.715435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:31.715901 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:31.716089 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:31.716659 (+ 570us) server_negotiation.cc:299] Negotiation successful
0808 11:44:31.716937 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 11:44:31.718655 22876 tablet_copy_service.cc:140] P 560789189ca9482d8310a358e6abfd54: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 365c520c6f164df1a8cf66562b8bea8c ({username='oryx', principal='oryx/127.13.158.124@KRBTEST.COM'} at 127.0.0.1:54272)
I20260808 11:44:31.718935 22876 tablet_copy_service.cc:161] P 560789189ca9482d8310a358e6abfd54: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 365c520c6f164df1a8cf66562b8bea8c at {username='oryx', principal='oryx/127.13.158.124@KRBTEST.COM'} at 127.0.0.1:54272: session id = 365c520c6f164df1a8cf66562b8bea8c-00000000000000000000000000000000
I20260808 11:44:31.723835 22876 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 11:44:31.724402 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 22934
I20260808 11:44:31.726063 23615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 11:44:31.732821 23615 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: tablet copy: Starting download of 0 data blocks...
I20260808 11:44:31.733176 23615 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: tablet copy: Starting download of 1 WAL segments...
I20260808 11:44:31.735670 23615 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 11:44:31.739426 22876 tablet_copy_service.cc:342] P 560789189ca9482d8310a358e6abfd54: Request end of tablet copy session 365c520c6f164df1a8cf66562b8bea8c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.158.124@KRBTEST.COM'} at 127.0.0.1:54272
I20260808 11:44:31.739578 22876 tablet_copy_service.cc:433] P 560789189ca9482d8310a358e6abfd54: ending tablet copy session 365c520c6f164df1a8cf66562b8bea8c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 365c520c6f164df1a8cf66562b8bea8c
I20260808 11:44:31.751430 23615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 11:44:31.753113 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:38839
--local_ip_for_outbound_sockets=127.13.158.65
--tserver_master_addrs=127.13.158.126:45849
--webserver_port=43199
--webserver_interface=127.13.158.65
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab
--principal=oryx/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621,127.13.158.124:39083 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:31.754530 23898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:31.763413 23899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:31.764809 23901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:31.767784 23615 server_base.cc:1034] running on GCE node
I20260808 11:44:31.768218 23615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:31.768978 23615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:31.770208 23615 hybrid_clock.cc:648] HybridClock initialized: now 1786189471770157 us; error 33 us; skew 500 ppm
I20260808 11:44:31.770498 23615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:31 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189471, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:31.772820 23615 init.cc:377] Logged in from keytab as oryx/127.13.158.124@KRBTEST.COM (short username oryx)
I20260808 11:44:31.780422 23615 webserver.cc:460] Webserver started at http://127.13.158.124:45801/ using document root <none> and password file <none>
I20260808 11:44:31.780830 23615 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:31.780946 23615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:31.784778 23615 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 11:44:31.786942 23912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:31.787397 23615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 11:44:31.787550 23615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
uuid: "365c520c6f164df1a8cf66562b8bea8c"
format_stamp: "Formatted at 2026-08-08 11:44:31 on dist-test-slave-39l8"
server_key: "dc92d46bb0bdfb0c20dd1b6cb7487456"
server_key_iv: "f3d34772b2c72f2730b8870bc5bfeaa2"
server_key_version: "encryptionkey@0"
I20260808 11:44:31.787806 23615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:31.817375 23615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:31.821679 23615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:31.844676 23615 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.124:39083
I20260808 11:44:31.844717 23977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.124:39083 every 8 connection(s)
I20260808 11:44:31.846148 23615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
I20260808 11:44:31.851441 23978 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:44:31.855278 23978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Bootstrap starting.
I20260808 11:44:31.872411 23978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:44:31.872958 23978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Bootstrap complete.
I20260808 11:44:31.874058 23978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } }
I20260808 11:44:31.874365 23978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 365c520c6f164df1a8cf66562b8bea8c, State: Initialized, Role: LEARNER
I20260808 11:44:31.874658 23978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } }
I20260808 11:44:31.875787 23978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:31.875536 23984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:31.876394 23984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: This master's current role is: LEARNER
I20260808 11:44:31.883185 23997 catalog_manager.cc:1259] Loaded cluster ID: d7473c94efbd4a0085f2b819a32e48a1
I20260808 11:44:31.883353 23997 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: loading cluster ID for follower catalog manager: success
I20260808 11:44:31.887099 23997 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: acquiring CA information for follower catalog manager: success
I20260808 11:44:31.889523 23997 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:44:32.035745 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.024819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.0.0.1:43686)
0808 11:44:32.025085 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:32.025103 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:32.025219 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:32.028768 (+ 3549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:32.028773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:32.028807 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:32.028976 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:32.028987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.030907 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.030915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:32.033129 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:32.033150 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.034813 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.034822 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:32.035070 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.035122 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 11:44:32.035224 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 11:44:32.035384 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260808 11:44:32.035768 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.025047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.13.158.124:39083)
0808 11:44:32.028354 (+ 3307us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:32.028359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:32.028383 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:44:32.028459 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:32.028464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:32.028583 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:32.028706 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:32.029173 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.030763 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.033317 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.034623 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.034843 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.035001 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:32.035261 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 11:44:32.035406 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3157,"thread_start_us":2926,"threads_started":1}
I20260808 11:44:32.037279 23946 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260808 11:44:32.037973 23984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 560789189ca9482d8310a358e6abfd54. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:32.038317 23984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: This master's current role is: LEARNER
W20260808 11:44:32.202857 23903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:32.203213 23903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:32.203464 23903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:32.213330 23903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:32.213537 23903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:32.213761 23903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:32.224462 23903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab
--principal=oryx/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:38839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=43199
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621,127.13.158.124:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:32.225953 23903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:32.227548 23903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:32.250872 24010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:32.254571 23903 server_base.cc:1034] running on GCE node
W20260808 11:44:32.255254 24013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:32.255550 24011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:32.256634 23903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:32.258122 23903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:32.259387 23903 hybrid_clock.cc:648] HybridClock initialized: now 1786189472259325 us; error 43 us; skew 500 ppm
I20260808 11:44:32.259756 23903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:32 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189472, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:32.264742 23903 init.cc:377] Logged in from keytab as oryx/127.13.158.65@KRBTEST.COM (short username oryx)
I20260808 11:44:32.267995 23903 webserver.cc:460] Webserver started at http://127.13.158.65:43199/ using document root <none> and password file <none>
I20260808 11:44:32.269363 23903 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:32.269692 23903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:32.278921 23903 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:44:32.283042 24023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:32.284462 23903 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 11:44:32.284591 23903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "3f196183a33d49499432ed6622045abf"
format_stamp: "Formatted at 2026-08-08 11:44:28 on dist-test-slave-39l8"
server_key: "395fcd2df2ba463a0c1ee9c129cbe454"
server_key_iv: "9173820f47776cced8e4b7ba937959ce"
server_key_version: "encryptionkey@0"
I20260808 11:44:32.285530 23903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:32.328190 23903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:32.334158 23903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:32.334563 23903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:32.335928 23903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:32.337396 23903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:32.337484 23903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:32.337599 23903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:32.337728 23903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:32.411716 23903 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:38839
I20260808 11:44:32.411936 24169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:38839 every 8 connection(s)
I20260808 11:44:32.413518 23903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:32.415485 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 23903
I20260808 11:44:32.415961 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 23108
Aug 08 11:44:32 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.66@KRBTEST.COM
Aug 08 11:44:32 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189472, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.125@KRBTEST.COM
Aug 08 11:44:32 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189472, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
Aug 08 11:44:32 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189472, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.124@KRBTEST.COM
I20260808 11:44:32.470606 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.418592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.13.158.66:41193)
0808 11:44:32.434175 (+ 15583us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:32.434188 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:32.439430 (+ 5242us) server_negotiation.cc:407] Connection header received
0808 11:44:32.439824 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:32.439839 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:32.440229 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:32.440409 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:32.444480 (+ 4071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.446199 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.448514 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.449914 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.450053 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.458094 (+ 8041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:32.458115 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:32.458135 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:32.458181 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:32.465760 (+ 7579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:32.467230 (+ 1470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:32.467242 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:32.467246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:32.467466 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:32.468104 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:32.468112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:32.468117 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:32.468599 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:32.468981 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:32.469705 (+ 724us) server_negotiation.cc:299] Negotiation successful
0808 11:44:32.470040 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15316,"thread_start_us":15097,"threads_started":1}
I20260808 11:44:32.473697 23277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:32.474833 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.420783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:49583 (local address 127.13.158.125:44621)
0808 11:44:32.432904 (+ 12121us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:32.432912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:32.443545 (+ 10633us) server_negotiation.cc:407] Connection header received
0808 11:44:32.443843 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:32.443848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:32.443942 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:32.444013 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:32.451231 (+ 7218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.452682 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.455118 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.455930 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.456019 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.464595 (+ 8576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:32.464602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:32.464607 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:32.464649 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:32.469791 (+ 5142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:32.470983 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:32.470988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:32.470992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:32.471197 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:32.471841 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:32.471845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:32.471848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:32.472416 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:32.472578 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:32.473829 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0808 11:44:32.474131 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11973,"thread_start_us":11727,"threads_started":1}
I20260808 11:44:32.475808 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.417918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:38889 (local address 127.13.158.126:45849)
0808 11:44:32.430379 (+ 12461us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:32.430389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:32.433637 (+ 3248us) server_negotiation.cc:407] Connection header received
0808 11:44:32.440575 (+ 6938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:32.440581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:32.440761 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:32.440864 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:32.447738 (+ 6874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.448983 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.458862 (+ 9879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.460020 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.460171 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.465228 (+ 5057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:32.465235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:32.465243 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:32.465285 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:32.471323 (+ 6038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:32.472479 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:32.472485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:32.472488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:32.472663 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:32.473406 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:32.473413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:32.473419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:32.473938 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:32.474070 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:32.474758 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 11:44:32.475260 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12281,"thread_start_us":13476,"threads_started":1}
I20260808 11:44:32.475786 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.417792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:45747 (local address 127.13.158.124:39083)
0808 11:44:32.419063 (+ 1271us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:32.419073 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:32.440470 (+ 21397us) server_negotiation.cc:407] Connection header received
0808 11:44:32.443213 (+ 2743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:32.443221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:32.443318 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:32.443435 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:32.449172 (+ 5737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.450592 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.455935 (+ 5343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.456881 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.457017 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.465581 (+ 8564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:32.465588 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:32.465593 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:32.465660 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:32.471323 (+ 5663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:32.472630 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:32.472636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:32.472639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:32.472902 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:32.473613 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:32.473618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:32.473621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:32.474060 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:32.474256 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:32.474901 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 11:44:32.475230 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1085}
I20260808 11:44:32.478204 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.418030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.65:38889)
0808 11:44:32.433387 (+ 15357us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:32.433465 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:32.440148 (+ 6683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:32.441570 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:32.441578 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:32.445504 (+ 3926us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:32.447549 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:32.447578 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.449326 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.449332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:32.458637 (+ 9305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:32.458651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.460676 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.460685 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:32.460931 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.461570 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:32.461598 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:32.464940 (+ 3342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:32.471498 (+ 6558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:32.471503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:32.471507 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:32.472330 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:32.472867 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:32.472872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:32.472875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:32.473147 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:32.474154 (+ 1007us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:32.474159 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:32.474359 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 11:44:32.476413 (+ 2054us) client_negotiation.cc:240] Negotiation successful
0808 11:44:32.476790 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14419,"mutex_wait_us":4767,"thread_start_us":21506,"threads_started":1}
I20260808 11:44:32.478596 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.418013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.13.158.65:45747)
0808 11:44:32.440350 (+ 22337us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:32.440369 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:32.442896 (+ 2527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:32.443744 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:32.443751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:32.445970 (+ 2219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:32.448862 (+ 2892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:32.448879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.450785 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.450791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:32.455753 (+ 4962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:32.455774 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.457450 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.457459 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:32.457791 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.461754 (+ 3963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:32.461775 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:32.465329 (+ 3554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:32.471507 (+ 6178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:32.471510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:32.471514 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:32.472495 (+ 981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:32.473056 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:32.473061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:32.473064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:32.473477 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:32.474344 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:32.474349 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:32.474543 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 11:44:32.476111 (+ 1568us) client_negotiation.cc:240] Negotiation successful
0808 11:44:32.476453 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22099,"mutex_wait_us":597,"spinlock_wait_cycles":55936,"thread_start_us":13324,"threads_started":1}
I20260808 11:44:32.478889 24188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:32.419900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.13.158.65:49583)
0808 11:44:32.443435 (+ 23535us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:32.443466 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:32.443673 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:32.444236 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:32.444241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:32.446341 (+ 2100us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:32.451025 (+ 4684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:32.451041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.452916 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.452922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:32.454957 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:32.454972 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:32.456487 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:32.456499 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:32.456819 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:32.457789 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:32.457858 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:32.464245 (+ 6387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:32.469957 (+ 5712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:32.469981 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:32.470000 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:32.470832 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:32.471349 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:32.471354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:32.471358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:32.471693 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:32.472652 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:32.472660 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:32.473194 (+ 534us) client_negotiation.cc:769] Sending connection context
0808 11:44:32.475192 (+ 1998us) client_negotiation.cc:240] Negotiation successful
0808 11:44:32.475897 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23405,"thread_start_us":22802,"threads_started":1}
I20260808 11:44:32.480443 24170 heartbeater.cc:347] Connected to a master server at 127.13.158.124:39083
I20260808 11:44:32.480778 24170 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:32.481590 24171 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45849
I20260808 11:44:32.481673 24172 heartbeater.cc:347] Connected to a master server at 127.13.158.125:44621
I20260808 11:44:32.481839 24172 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:32.481839 24171 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:32.483619 24170 heartbeater.cc:511] Master 127.13.158.124:39083 requested a full tablet report, sending...
I20260808 11:44:32.485411 24171 heartbeater.cc:511] Master 127.13.158.126:45849 requested a full tablet report, sending...
I20260808 11:44:32.486536 23935 ts_manager.cc:194] Registered new tserver with Master: 3f196183a33d49499432ed6622045abf (127.13.158.65:38839)
I20260808 11:44:32.486842 24172 heartbeater.cc:511] Master 127.13.158.125:44621 requested a full tablet report, sending...
I20260808 11:44:32.487159 22850 ts_manager.cc:194] Re-registered known tserver with Master: 3f196183a33d49499432ed6622045abf (127.13.158.65:38839)
I20260808 11:44:32.488870 22850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:38889
I20260808 11:44:32.491009 23672 ts_manager.cc:194] Registered new tserver with Master: 3f196183a33d49499432ed6622045abf (127.13.158.65:38839)
I20260808 11:44:32.556382 23886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: NON_VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: true } } }
I20260808 11:44:32.556885 23886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:32.559301 23886 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: attempting to promote NON_VOTER 365c520c6f164df1a8cf66562b8bea8c to VOTER
I20260808 11:44:32.560009 23886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } }
I20260808 11:44:32.562005 23946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 LEARNER]: Refusing update from remote peer 560789189ca9482d8310a358e6abfd54: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 11:44:32.562040 23687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Refusing update from remote peer 560789189ca9482d8310a358e6abfd54: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 11:44:32.562728 24192 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 11:44:32.563365 24193 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 11:44:32.566448 24192 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 365c520c6f164df1a8cf66562b8bea8c (127.13.158.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:32.567508 23687 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 365c520c6f164df1a8cf66562b8bea8c (127.13.158.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:32.572158 23886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:32.572402 23886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:32.572701 24192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:32.572957 24192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:32.576972 24197 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:32.577802 24197 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:44:32.579293 23886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:32.579722 23886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:32.580176 23946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 365c520c6f164df1a8cf66562b8bea8c (127.13.158.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:32.581954 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 23108
I20260808 11:44:32.589818 24199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "560789189ca9482d8310a358e6abfd54" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:32.590224 24199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:44:32.604841 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:41193
--local_ip_for_outbound_sockets=127.13.158.66
--tserver_master_addrs=127.13.158.126:45849
--webserver_port=43375
--webserver_interface=127.13.158.66
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab
--principal=oryx/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621,127.13.158.124:39083 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:32.974380 24200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:32.974689 24200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:32.974884 24200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:32.983165 24200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:32.983310 24200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:32.983489 24200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:32.994190 24200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab
--principal=oryx/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:41193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=43375
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621,127.13.158.124:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:32.995601 24200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:32.997165 24200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:33.012537 24206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:33.012718 24208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:33.013336 24205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:33.018684 24200 server_base.cc:1034] running on GCE node
I20260808 11:44:33.019425 24200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:33.020445 24200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:33.021749 24200 hybrid_clock.cc:648] HybridClock initialized: now 1786189473021668 us; error 70 us; skew 500 ppm
I20260808 11:44:33.022105 24200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189473, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:33.025830 24200 init.cc:377] Logged in from keytab as oryx/127.13.158.66@KRBTEST.COM (short username oryx)
I20260808 11:44:33.027902 24200 webserver.cc:460] Webserver started at http://127.13.158.66:43375/ using document root <none> and password file <none>
I20260808 11:44:33.028682 24200 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:33.028836 24200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:33.034587 24200 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 11:44:33.037618 24216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:33.039220 24200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:33.039409 24200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "2ff970853d2f47bdb3eecc00a9310ec4"
format_stamp: "Formatted at 2026-08-08 11:44:29 on dist-test-slave-39l8"
server_key: "e1f28da827f504cab5717b0a79f7af3e"
server_key_iv: "ec7632ab21e18e3b33f9213c9a692d3c"
server_key_version: "encryptionkey@0"
I20260808 11:44:33.040412 24200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:33.053313 24200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:33.057508 24200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:33.057956 24200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:33.059008 24200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:33.060508 24200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:33.060636 24200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:33.060772 24200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:33.060877 24200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:33.105492 24200 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:41193
I20260808 11:44:33.105546 24370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:41193 every 8 connection(s)
I20260808 11:44:33.106966 24200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:33.109052 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 24200
I20260808 11:44:33.109372 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 23290
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189473, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.125@KRBTEST.COM
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189473, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.67@KRBTEST.COM
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189473, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.124@KRBTEST.COM
I20260808 11:44:33.139776 24381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.111360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:45893 (local address 127.13.158.126:45849)
0808 11:44:33.119543 (+ 8183us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.119550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.120653 (+ 1103us) server_negotiation.cc:407] Connection header received
0808 11:44:33.120711 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.120715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.120784 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.120902 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.122446 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.123506 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.125186 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.125886 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.126036 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.131558 (+ 5522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.131565 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.131570 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.131617 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.135536 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.136557 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.136565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.136570 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.136797 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.137541 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.137546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.137549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.137882 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.138033 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.139191 (+ 1158us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.139410 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8052,"thread_start_us":7968,"threads_started":1}
I20260808 11:44:33.140825 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.110802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.13.158.66:60965)
0808 11:44:33.111936 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.112020 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.115293 (+ 3273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.120264 (+ 4971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.120277 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.121168 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:33.122134 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.122168 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.123796 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.123803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.125435 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.125450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.126649 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.126655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.126851 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.127441 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:33.127460 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:33.131253 (+ 3793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:33.135553 (+ 4300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.135562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.135586 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.137016 (+ 1430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.137400 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.137405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.137407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.137603 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.138274 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:33.138278 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:33.138469 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.138819 (+ 350us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.140389 (+ 1570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":1101,"spinlock_wait_cycles":23936,"thread_start_us":7824,"threads_started":1}
I20260808 11:44:33.140851 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.110802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.66:45893)
0808 11:44:33.120006 (+ 9204us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.120035 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.120180 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.120923 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.120927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.121740 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:33.122295 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.122312 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.123626 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.123631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.125034 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.125046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.126088 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.126097 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.126323 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.127040 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:33.127065 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:33.130677 (+ 3612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:33.135658 (+ 4981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.135662 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.135666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.136433 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.137169 (+ 736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.137175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.137179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.137419 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.138023 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:33.138032 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:33.138446 (+ 414us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.138819 (+ 373us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.139369 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9029,"mutex_wait_us":4828,"spinlock_wait_cycles":14720,"thread_start_us":362,"threads_started":1}
I20260808 11:44:33.141981 24383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.112571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:60965 (local address 127.13.158.125:44621)
0808 11:44:33.119705 (+ 7134us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.119710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.119726 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:44:33.119791 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.119795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.119855 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.119985 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.122391 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.123604 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.125634 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.126286 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.126401 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.131472 (+ 5071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.131479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.131485 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.131524 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.135385 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.137133 (+ 1748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.137137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.137139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.137262 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.137743 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.137751 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.137756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.138135 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.138283 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.141358 (+ 3075us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.141606 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7009,"thread_start_us":6915,"threads_started":1}
I20260808 11:44:33.143266 24375 heartbeater.cc:347] Connected to a master server at 127.13.158.125:44621
I20260808 11:44:33.143594 24375 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:33.144500 24375 heartbeater.cc:511] Master 127.13.158.125:44621 requested a full tablet report, sending...
I20260808 11:44:33.145355 24372 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45849
I20260808 11:44:33.145485 24372 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:33.145936 24372 heartbeater.cc:511] Master 127.13.158.126:45849 requested a full tablet report, sending...
I20260808 11:44:33.146754 23672 ts_manager.cc:194] Registered new tserver with Master: 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66:41193)
I20260808 11:44:33.146855 22850 ts_manager.cc:194] Re-registered known tserver with Master: 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66:41193)
I20260808 11:44:33.147169 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.110802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.13.158.66:43367)
0808 11:44:33.116187 (+ 5385us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.116201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.116283 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.123412 (+ 7129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.123417 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.125648 (+ 2231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:33.126601 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.126626 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.130380 (+ 3754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.130388 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.131829 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.131845 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.133975 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.133982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.134168 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.134787 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:33.134812 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:33.139173 (+ 4361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:33.143395 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.143402 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.143408 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.144079 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.144614 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.144623 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.144629 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.144833 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.145738 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:33.145745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:33.145990 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.146157 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.146550 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5257,"mutex_wait_us":4412,"thread_start_us":4416,"threads_started":1}
I20260808 11:44:33.147979 22850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:45893
I20260808 11:44:33.148711 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.111216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:43367 (local address 127.13.158.124:39083)
0808 11:44:33.123082 (+ 11866us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.123088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.123107 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:44:33.123163 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.123167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.123246 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.126829 (+ 3583us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.129074 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.130216 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.132017 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.133129 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.133277 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.139410 (+ 6133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.139419 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.139427 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.139473 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.143159 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.144257 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.144265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.144271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.144439 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.144985 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.144993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.144999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.145462 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.145643 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.147958 (+ 2315us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.148272 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11725,"thread_start_us":12381,"threads_started":1}
I20260808 11:44:33.148749 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.113302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.13.158.67:40381)
0808 11:44:33.120626 (+ 7324us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.120636 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.120668 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:44:33.120740 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.120747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.120986 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.121304 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.123683 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.125280 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.127418 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.128265 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.128415 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.134478 (+ 6063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.134495 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.134513 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.134574 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.138336 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.139203 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.139211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.139214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.139379 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.139836 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.139841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.139845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.140301 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.140602 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.141936 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.142226 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7138,"thread_start_us":6988,"threads_started":1}
I20260808 11:44:33.149871 24371 heartbeater.cc:347] Connected to a master server at 127.13.158.124:39083
I20260808 11:44:33.150028 24371 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:33.150137 23465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:33.150483 24371 heartbeater.cc:511] Master 127.13.158.124:39083 requested a full tablet report, sending...
I20260808 11:44:33.151947 23935 ts_manager.cc:194] Registered new tserver with Master: 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66:41193)
I20260808 11:44:33.236770 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 23290
I20260808 11:44:33.259244 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:40381
--local_ip_for_outbound_sockets=127.13.158.67
--tserver_master_addrs=127.13.158.126:45849
--webserver_port=40945
--webserver_interface=127.13.158.67
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab
--principal=oryx/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621,127.13.158.124:39083 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:44:33.491356 24171 heartbeater.cc:503] Master 127.13.158.126:45849 was elected leader, sending a full tablet report...
W20260808 11:44:33.602298 24395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:33.602638 24395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:33.602862 24395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:33.610258 24395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:33.610400 24395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:33.610570 24395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:33.621218 24395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab
--principal=oryx/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:40381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=40945
--tserver_master_addrs=127.13.158.126:45849,127.13.158.125:44621,127.13.158.124:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:33.622570 24395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:33.623904 24395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:33.638481 24400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:33.638413 24401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:33.638450 24404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:33.639317 24395 server_base.cc:1034] running on GCE node
I20260808 11:44:33.639863 24395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:33.640785 24395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:33.642035 24395 hybrid_clock.cc:648] HybridClock initialized: now 1786189473641975 us; error 39 us; skew 500 ppm
I20260808 11:44:33.642488 24395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189473, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:33.645938 24395 init.cc:377] Logged in from keytab as oryx/127.13.158.67@KRBTEST.COM (short username oryx)
I20260808 11:44:33.647436 24395 webserver.cc:460] Webserver started at http://127.13.158.67:40945/ using document root <none> and password file <none>
I20260808 11:44:33.648075 24395 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:33.648191 24395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:33.653568 24395 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:44:33.656386 24415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:33.657691 24395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:44:33.657861 24395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "964f9eaed9c843cd8ef86ee7c526a783"
format_stamp: "Formatted at 2026-08-08 11:44:29 on dist-test-slave-39l8"
server_key: "139988b82c9ad00bdc40ca2511dada50"
server_key_iv: "e2277de9e7103fbcbb75bb12b4ed57c8"
server_key_version: "encryptionkey@0"
I20260808 11:44:33.658877 24395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:33.681773 24395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:33.686206 24395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:33.686441 24395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:33.687492 24395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:33.689208 24395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:33.689325 24395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:33.689409 24395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:33.689464 24395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:33.722689 24395 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:40381
I20260808 11:44:33.722846 24574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:40381 every 8 connection(s)
I20260808 11:44:33.724185 24395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:33.732546 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 24395
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189473, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189473, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.124@KRBTEST.COM
I20260808 11:44:33.768872 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.727617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.67:46227)
0808 11:44:33.742215 (+ 14598us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.742251 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.743255 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.744324 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.744334 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.745680 (+ 1346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:33.747043 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.747066 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.748650 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.748656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.751354 (+ 2698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.751373 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.752718 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.752728 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.753069 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.753970 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:33.754023 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:33.757947 (+ 3924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:33.763300 (+ 5353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.763315 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.763339 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.764073 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.764559 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.764567 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.764573 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.764829 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.765664 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:33.765675 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:33.766169 (+ 494us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.766513 (+ 344us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.767164 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14394,"mutex_wait_us":429,"spinlock_wait_cycles":2432,"thread_start_us":14021,"threads_started":1}
I20260808 11:44:33.770102 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.728655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:46227 (local address 127.13.158.126:45849)
0808 11:44:33.735207 (+ 6552us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.735217 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.742373 (+ 7156us) server_negotiation.cc:407] Connection header received
0808 11:44:33.743704 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.743710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.743793 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.743885 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.747286 (+ 3401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.748455 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.751544 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.752220 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.752362 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.758238 (+ 5876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.758247 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.758255 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.758306 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.762978 (+ 4672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.764239 (+ 1261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.764248 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.764254 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.764418 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.764978 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.764987 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.764992 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.765422 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.765582 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.769261 (+ 3679us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.769581 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6371,"thread_start_us":6289,"threads_started":1}
I20260808 11:44:33.772660 24576 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45849
I20260808 11:44:33.773072 24576 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:33.773720 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.728340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:48801 (local address 127.13.158.124:39083)
0808 11:44:33.735681 (+ 7341us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.735690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.740071 (+ 4381us) server_negotiation.cc:407] Connection header received
0808 11:44:33.743678 (+ 3607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.743683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.743773 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.743877 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.747276 (+ 3399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.748640 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.752257 (+ 3617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.753085 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.753251 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.763621 (+ 10370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.763632 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.763637 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.763678 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.767910 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.768841 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.768846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.768849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.769085 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.770470 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.770478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.770484 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.770856 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.771018 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.773033 (+ 2015us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.773307 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7187,"thread_start_us":7052,"threads_started":1}
I20260808 11:44:33.774060 24576 heartbeater.cc:511] Master 127.13.158.126:45849 requested a full tablet report, sending...
Aug 08 11:44:33 dist-test-slave-39l8 krb5kdc[22785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189473, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.125@KRBTEST.COM
I20260808 11:44:33.776147 22850 ts_manager.cc:194] Re-registered known tserver with Master: 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381)
I20260808 11:44:33.777276 22850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:46227
I20260808 11:44:33.782002 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.727616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.13.158.67:48801)
0808 11:44:33.739853 (+ 12237us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.739915 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.743256 (+ 3341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.744321 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.744332 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.745684 (+ 1352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:33.747045 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.747069 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.748840 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.748848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.752024 (+ 3176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.752061 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.754738 (+ 2677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.754747 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.755012 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.757050 (+ 2038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:33.757081 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:33.762466 (+ 5385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:33.768096 (+ 5630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.768100 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.768103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.768681 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.770026 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.770033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.770037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.770207 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.771402 (+ 1195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:33.771406 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:33.772164 (+ 758us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.772310 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.774552 (+ 2242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11850,"mutex_wait_us":1073,"spinlock_wait_cycles":58240,"thread_start_us":12530,"threads_started":1}
I20260808 11:44:33.783367 24575 heartbeater.cc:347] Connected to a master server at 127.13.158.124:39083
I20260808 11:44:33.783535 24575 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:33.784240 24575 heartbeater.cc:511] Master 127.13.158.124:39083 requested a full tablet report, sending...
I20260808 11:44:33.785998 23935 ts_manager.cc:194] Registered new tserver with Master: 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381)
I20260808 11:44:33.788255 24580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.727021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:53147 (local address 127.13.158.125:44621)
0808 11:44:33.741608 (+ 14587us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.741614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.741643 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:44:33.743615 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.743621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.743709 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.743791 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.747504 (+ 3713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.748490 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.771058 (+ 22568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.771721 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.771812 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.777514 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.777523 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.777530 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.777588 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.783792 (+ 6204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.784806 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.784811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.784814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.785075 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.785772 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.785777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.785780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.786329 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.786466 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.787360 (+ 894us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.787641 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14409,"thread_start_us":14308,"threads_started":1}
I20260808 11:44:33.788548 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.727616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.13.158.67:53147)
0808 11:44:33.740636 (+ 13020us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.740664 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.743250 (+ 2586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.745481 (+ 2231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.745485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.746117 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:33.747061 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.747069 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.748684 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.748690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.770726 (+ 22036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.770742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.772159 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.772167 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.772358 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.773407 (+ 1049us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:33.773430 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:33.777241 (+ 3811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:33.784003 (+ 6762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.784034 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.784040 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.784662 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.785247 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:33.785253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:33.785256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:33.785572 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:33.786575 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:33.786583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:33.786886 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.787082 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.787604 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12831,"mutex_wait_us":966,"spinlock_wait_cycles":46464,"thread_start_us":11561,"threads_started":1}
I20260808 11:44:33.789850 24578 heartbeater.cc:347] Connected to a master server at 127.13.158.125:44621
I20260808 11:44:33.790047 24578 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:33.790691 24578 heartbeater.cc:511] Master 127.13.158.125:44621 requested a full tablet report, sending...
I20260808 11:44:33.792464 23672 ts_manager.cc:194] Registered new tserver with Master: 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381)
I20260808 11:44:33.803615 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.745870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.13.158.125:44621)
0808 11:44:33.760979 (+ 15109us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.760988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.762472 (+ 1484us) server_negotiation.cc:407] Connection header received
0808 11:44:33.763972 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.763978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.764062 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.764158 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.766783 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.768121 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.777389 (+ 9268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.778142 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.778245 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.793357 (+ 15112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.793366 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.793371 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.793421 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.798537 (+ 5116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.799825 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.799841 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.799846 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.800162 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.800724 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.800730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.800734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.801227 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.801604 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.802134 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.802677 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14930,"thread_start_us":15122,"threads_started":1}
I20260808 11:44:33.805243 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.744604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.13.158.126:45849)
0808 11:44:33.770516 (+ 25912us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.770523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.770547 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:44:33.770607 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.770615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.770737 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.770838 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.774672 (+ 3834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.775768 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.789413 (+ 13645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.790404 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.790539 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.796093 (+ 5554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.796102 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.796109 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.796154 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.800591 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.801523 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.801529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.801532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.801744 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.802344 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.802349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.802352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.802840 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.803152 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.803607 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.803937 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25775,"thread_start_us":44234,"threads_started":1}
I20260808 11:44:33.807945 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.752177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.13.158.124:39083)
0808 11:44:33.781062 (+ 28885us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.781068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.781087 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:44:33.781169 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.781174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.781304 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.781440 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:33.788846 (+ 7406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.790414 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.793893 (+ 3479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.795023 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.795192 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.799725 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:33.799730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:33.799734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:33.799799 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:33.804563 (+ 4764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.805305 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.805310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.805312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.805556 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:33.805979 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:33.805984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:33.805988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:33.806310 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:33.806615 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.807197 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.807535 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28749,"thread_start_us":34903,"threads_started":1}
I20260808 11:44:33.810503 22850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:44:33.819976 22850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:33.889431 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.846821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:38839 (local address 127.0.0.1:47864)
0808 11:44:33.869562 (+ 22741us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.869618 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.874930 (+ 5312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.876476 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.876481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.876510 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:33.877607 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.877628 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.881065 (+ 3437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.881070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.885406 (+ 4336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.885428 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.888190 (+ 2762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.888205 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.888486 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.888522 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.888607 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.888777 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22487,"thread_start_us":30354,"threads_started":1}
I20260808 11:44:33.890971 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.847175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.13.158.65:38839)
0808 11:44:33.875797 (+ 28622us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.875811 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.875839 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:44:33.875929 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.875935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.876211 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.876458 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:33.877851 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.880735 (+ 2884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.885674 (+ 4939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.887894 (+ 2220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.889813 (+ 1919us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.890083 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.890157 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.890565 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28435,"thread_start_us":28298,"threads_started":1}
I20260808 11:44:33.892506 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.847326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41193 (local address 127.0.0.1:47768)
0808 11:44:33.877633 (+ 30307us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.877654 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.877800 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.878797 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.878801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.878837 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:33.879580 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.879596 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.882233 (+ 2637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.882240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.888337 (+ 6097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.888351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.891505 (+ 3154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.891513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.891732 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.891763 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.891852 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.891971 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30121,"mutex_wait_us":1,"thread_start_us":22480,"threads_started":1}
I20260808 11:44:33.892608 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.847641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.13.158.66:41193)
0808 11:44:33.871509 (+ 23868us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.871523 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.877942 (+ 6419us) server_negotiation.cc:407] Connection header received
0808 11:44:33.878075 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.878082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.878349 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.878593 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:33.879943 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.881940 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.888512 (+ 6572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.890455 (+ 1943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.890562 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.891131 (+ 569us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.891949 (+ 818us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.892191 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23600,"thread_start_us":23377,"threads_started":1}
I20260808 11:44:33.896207 24084 tablet_service.cc:1467] Processing CreateTablet for tablet 73194cc434db46b5a20d00731e050b76 (DEFAULT_TABLE table=test-table [id=2ceb469d16384f0eb2cbe4d320649370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:33.896620 24282 tablet_service.cc:1467] Processing CreateTablet for tablet 73194cc434db46b5a20d00731e050b76 (DEFAULT_TABLE table=test-table [id=2ceb469d16384f0eb2cbe4d320649370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:33.898022 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.870396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:40381 (local address 127.0.0.1:46670)
0808 11:44:33.882983 (+ 12587us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.883005 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.883158 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.885254 (+ 2096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.885259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.885283 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:33.886406 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.886419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.890644 (+ 4225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.890649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.894499 (+ 3850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.894511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.896659 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.896668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.896983 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.897006 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.897115 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.897250 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12442,"mutex_wait_us":44,"thread_start_us":11933,"threads_started":1}
I20260808 11:44:33.898425 24084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73194cc434db46b5a20d00731e050b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:33.898638 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.870871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.13.158.67:40381)
0808 11:44:33.883617 (+ 12746us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.883626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.883674 (+ 48us) server_negotiation.cc:407] Connection header received
0808 11:44:33.884092 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.884101 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.884347 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.884586 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:33.886949 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.890414 (+ 3465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.894645 (+ 4231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.896284 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.896471 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.896749 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.897169 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.897552 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12489,"thread_start_us":12329,"threads_started":1}
I20260808 11:44:33.898797 24282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73194cc434db46b5a20d00731e050b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:33.902298 24483 tablet_service.cc:1467] Processing CreateTablet for tablet 73194cc434db46b5a20d00731e050b76 (DEFAULT_TABLE table=test-table [id=2ceb469d16384f0eb2cbe4d320649370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:33.903556 24483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73194cc434db46b5a20d00731e050b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:33.923882 24623 tablet_bootstrap.cc:492] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: Bootstrap starting.
I20260808 11:44:33.925309 24625 tablet_bootstrap.cc:492] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: Bootstrap starting.
I20260808 11:44:33.928686 24624 tablet_bootstrap.cc:492] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: Bootstrap starting.
I20260808 11:44:33.932703 24623 tablet_bootstrap.cc:654] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:33.932790 24624 tablet_bootstrap.cc:654] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:33.933112 24625 tablet_bootstrap.cc:654] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:33.935168 24624 log.cc:824] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:33.935207 24623 log.cc:824] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:33.935380 24625 log.cc:824] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:33.939683 24623 tablet_bootstrap.cc:492] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: No bootstrap required, opened a new log
I20260808 11:44:33.940039 24623 ts_tablet_manager.cc:1397] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.001s
I20260808 11:44:33.940837 24624 tablet_bootstrap.cc:492] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: No bootstrap required, opened a new log
I20260808 11:44:33.941196 24624 ts_tablet_manager.cc:1397] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260808 11:44:33.945799 24625 tablet_bootstrap.cc:492] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: No bootstrap required, opened a new log
I20260808 11:44:33.946187 24625 ts_tablet_manager.cc:1397] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.008s
I20260808 11:44:33.955075 24623 raft_consensus.cc:348] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:33.955598 24623 raft_consensus.cc:374] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:33.955758 24623 raft_consensus.cc:729] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff970853d2f47bdb3eecc00a9310ec4, State: Initialized, Role: FOLLOWER
I20260808 11:44:33.955382 24624 raft_consensus.cc:348] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:33.956087 24624 raft_consensus.cc:374] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:33.956209 24624 raft_consensus.cc:729] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f196183a33d49499432ed6622045abf, State: Initialized, Role: FOLLOWER
I20260808 11:44:33.956730 24623 consensus_queue.cc:260] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:33.956990 24624 consensus_queue.cc:260] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:33.959167 24623 ts_tablet_manager.cc:1428] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: Time spent starting tablet: real 0.019s user 0.012s sys 0.007s
I20260808 11:44:33.959527 24372 heartbeater.cc:503] Master 127.13.158.126:45849 was elected leader, sending a full tablet report...
I20260808 11:44:33.966784 24624 ts_tablet_manager.cc:1428] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: Time spent starting tablet: real 0.025s user 0.018s sys 0.006s
I20260808 11:44:33.979204 24632 raft_consensus.cc:482] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:33.979561 24632 raft_consensus.cc:504] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:33.980418 24625 raft_consensus.cc:348] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:33.981426 24625 raft_consensus.cc:374] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:33.981730 24625 raft_consensus.cc:729] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964f9eaed9c843cd8ef86ee7c526a783, State: Initialized, Role: FOLLOWER
I20260808 11:44:33.982910 24625 consensus_queue.cc:260] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:33.987766 24632 leader_election.cc:288] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66:41193), 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381)
I20260808 11:44:33.992349 24576 heartbeater.cc:503] Master 127.13.158.126:45849 was elected leader, sending a full tablet report...
I20260808 11:44:33.993428 24625 ts_tablet_manager.cc:1428] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: Time spent starting tablet: real 0.047s user 0.017s sys 0.012s
I20260808 11:44:33.998422 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.986216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:33357 (local address 127.13.158.67:40381)
0808 11:44:33.986448 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.986454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.986472 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:44:33.986533 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.986547 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.986611 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.986696 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:33.988305 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.989602 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.996128 (+ 6526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.997319 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.997437 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.997519 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.998043 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.998173 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:44:33.998696 24635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.984376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:40381 (local address 127.13.158.65:33357)
0808 11:44:33.985490 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.985508 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.985673 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.986756 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.986763 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.986793 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:33.987749 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.987767 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.989929 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.989937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.995913 (+ 5976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.995930 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.997694 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.997699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.997853 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.997888 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.998001 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.998302 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"spinlock_wait_cycles":44928,"thread_start_us":3025,"threads_started":1}
I20260808 11:44:33.999853 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.982104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:36381 (local address 127.13.158.66:41193)
0808 11:44:33.982370 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:33.982377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:33.984490 (+ 2113us) server_negotiation.cc:407] Connection header received
0808 11:44:33.984912 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:33.984920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:33.985033 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:33.985143 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:33.986202 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.988764 (+ 2562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.996800 (+ 8036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.998695 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.998869 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.998946 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:33.999585 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 11:44:33.999684 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:44:33.999790 24510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73194cc434db46b5a20d00731e050b76" candidate_uuid: "3f196183a33d49499432ed6622045abf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964f9eaed9c843cd8ef86ee7c526a783" is_pre_election: true
I20260808 11:44:34.000176 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:33.982605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41193 (local address 127.13.158.65:36381)
0808 11:44:33.983778 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:33.984380 (+ 602us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:33.984674 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:33.985439 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:33.985444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:33.985484 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:33.985973 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.985990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.989038 (+ 3048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.989043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.996552 (+ 7509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:33.996569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:33.999163 (+ 2594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:33.999191 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:33.999385 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:33.999421 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:44:33.999543 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 11:44:33.999702 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"thread_start_us":1190,"threads_started":1}
I20260808 11:44:34.000603 24510 raft_consensus.cc:2454] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f196183a33d49499432ed6622045abf in term 0.
I20260808 11:44:34.001230 24309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73194cc434db46b5a20d00731e050b76" candidate_uuid: "3f196183a33d49499432ed6622045abf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" is_pre_election: true
I20260808 11:44:34.001417 24029 leader_election.cc:302] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f196183a33d49499432ed6622045abf, 964f9eaed9c843cd8ef86ee7c526a783; no voters:
I20260808 11:44:34.001770 24309 raft_consensus.cc:2454] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f196183a33d49499432ed6622045abf in term 0.
I20260808 11:44:34.001878 24632 raft_consensus.cc:2781] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:34.002146 24632 raft_consensus.cc:482] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:34.002315 24632 raft_consensus.cc:3037] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:34.005539 24632 raft_consensus.cc:504] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:34.006291 24632 leader_election.cc:288] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [CANDIDATE]: Term 1 election: Requested vote from peers 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66:41193), 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381)
I20260808 11:44:34.006755 24309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73194cc434db46b5a20d00731e050b76" candidate_uuid: "3f196183a33d49499432ed6622045abf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff970853d2f47bdb3eecc00a9310ec4"
I20260808 11:44:34.006935 24510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73194cc434db46b5a20d00731e050b76" candidate_uuid: "3f196183a33d49499432ed6622045abf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964f9eaed9c843cd8ef86ee7c526a783"
I20260808 11:44:34.007076 24309 raft_consensus.cc:3037] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:34.007201 24510 raft_consensus.cc:3037] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:34.011775 24309 raft_consensus.cc:2454] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f196183a33d49499432ed6622045abf in term 1.
I20260808 11:44:34.012414 24029 leader_election.cc:302] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff970853d2f47bdb3eecc00a9310ec4, 3f196183a33d49499432ed6622045abf; no voters:
I20260808 11:44:34.012733 24510 raft_consensus.cc:2454] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f196183a33d49499432ed6622045abf in term 1.
I20260808 11:44:34.012919 24632 raft_consensus.cc:2781] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:34.014133 24632 raft_consensus.cc:686] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 1 LEADER]: Becoming Leader. State: Replica: 3f196183a33d49499432ed6622045abf, State: Running, Role: LEADER
I20260808 11:44:34.015025 24632 consensus_queue.cc:237] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } }
I20260808 11:44:34.023110 22851 catalog_manager.cc:5495] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f196183a33d49499432ed6622045abf (127.13.158.65). New cstate: current_term: 1 leader_uuid: "3f196183a33d49499432ed6622045abf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } health_report { overall_health: UNKNOWN } } }
I20260808 11:44:34.055104 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:34.048857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47876 (local address 127.13.158.65:38839)
0808 11:44:34.049114 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:34.049119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:34.049134 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:44:34.049220 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:34.049223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:34.049335 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:34.049490 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:34.050063 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.050990 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.052635 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.053137 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.053288 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:34.053898 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:34.054466 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:34.054630 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:34.054730 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 11:44:34.054811 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:44:34.064587 24510 raft_consensus.cc:1261] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 1 FOLLOWER]: Refusing update from remote peer 3f196183a33d49499432ed6622045abf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:34.064805 24309 raft_consensus.cc:1261] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 FOLLOWER]: Refusing update from remote peer 3f196183a33d49499432ed6622045abf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:34.065654 24636 consensus_queue.cc:1035] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [LEADER]: Connected to new peer: Peer: permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:34.066188 24632 consensus_queue.cc:1035] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:34.080489 24640 mvcc.cc:205] Tried to move back new op lower bound from 7316232085758087168 to 7316232085571903488. Current Snapshot: MvccSnapshot[applied={T|T < 7316232085758087168}]
I20260808 11:44:34.101804 22851 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:34.102283 22851 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:34.112098 22851 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 560789189ca9482d8310a358e6abfd54: Sending DeleteTablet for 3 replicas of tablet 73194cc434db46b5a20d00731e050b76
I20260808 11:44:34.113616 24282 tablet_service.cc:1514] Processing DeleteTablet for tablet 73194cc434db46b5a20d00731e050b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:34 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:47768
I20260808 11:44:34.113943 24483 tablet_service.cc:1514] Processing DeleteTablet for tablet 73194cc434db46b5a20d00731e050b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:34 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:46670
I20260808 11:44:34.114719 24084 tablet_service.cc:1514] Processing DeleteTablet for tablet 73194cc434db46b5a20d00731e050b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:34 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:47864
I20260808 11:44:34.124490 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 22809
I20260808 11:44:34.125844 22890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:34.126137 24651 tablet_replica.cc:330] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: stopping tablet replica
I20260808 11:44:34.126677 24651 raft_consensus.cc:2229] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:34.127707 24651 raft_consensus.cc:2258] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:34.128531 24652 tablet_replica.cc:330] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: stopping tablet replica
I20260808 11:44:34.129112 24652 raft_consensus.cc:2229] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:34.129889 24652 raft_consensus.cc:2258] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:34.134814 24652 ts_tablet_manager.cc:1905] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:34.135118 24654 tablet_replica.cc:330] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: stopping tablet replica
I20260808 11:44:34.135716 24654 raft_consensus.cc:2229] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:44:34.136762 24654 raft_consensus.cc:2258] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:34.142696 24654 ts_tablet_manager.cc:1905] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:34.145789 24652 ts_tablet_manager.cc:1918] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:34.146153 24652 log.cc:1196] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/73194cc434db46b5a20d00731e050b76
I20260808 11:44:34.147198 24652 ts_tablet_manager.cc:1939] T 73194cc434db46b5a20d00731e050b76 P 964f9eaed9c843cd8ef86ee7c526a783: Deleting consensus metadata
I20260808 11:44:34.148481 24651 ts_tablet_manager.cc:1905] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:34.156077 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:34.117395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40928 (local address 127.13.158.125:44621)
0808 11:44:34.117595 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:34.117601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:34.123636 (+ 6035us) server_negotiation.cc:407] Connection header received
0808 11:44:34.123926 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:34.123933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:34.124054 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:34.124158 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:34.136964 (+ 12806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.138514 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.140959 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.141802 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.141990 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:34.146753 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:34.146761 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:34.146766 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:34.146811 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:34.152162 (+ 5351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:34.153263 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:34.153269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:34.153272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:34.153509 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:34.154040 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:34.154045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:34.154048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:34.154483 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:34.154917 (+ 434us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:34.155300 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 11:44:34.155599 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:44:34.160635 24651 ts_tablet_manager.cc:1918] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:34.160913 24651 log.cc:1196] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/73194cc434db46b5a20d00731e050b76
I20260808 11:44:34.161787 24651 ts_tablet_manager.cc:1939] T 73194cc434db46b5a20d00731e050b76 P 2ff970853d2f47bdb3eecc00a9310ec4: Deleting consensus metadata
I20260808 11:44:34.163897 24654 ts_tablet_manager.cc:1918] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:34.164202 24654 log.cc:1196] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/73194cc434db46b5a20d00731e050b76
I20260808 11:44:34.165902 24654 ts_tablet_manager.cc:1939] T 73194cc434db46b5a20d00731e050b76 P 3f196183a33d49499432ed6622045abf: Deleting consensus metadata
I20260808 11:44:34.182070 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:34.123529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.13.158.124:39083)
0808 11:44:34.123697 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:34.123702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:34.156078 (+ 32376us) server_negotiation.cc:407] Connection header received
0808 11:44:34.156230 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:34.156238 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:34.156322 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:34.156428 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:34.158959 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.160337 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.163348 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.164271 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.164409 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:34.174871 (+ 10462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:34.174878 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:34.174883 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:34.174930 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:34.178889 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:34.179806 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:34.179813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:34.179817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:34.179971 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:34.180414 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:34.180419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:34.180423 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:34.180791 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:34.181040 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:34.181457 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 11:44:34.181716 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:44:34.307667 22831 catalog_manager.cc:4841] TS 3f196183a33d49499432ed6622045abf (127.13.158.65:38839): tablet 73194cc434db46b5a20d00731e050b76 (table test-table [id=2ceb469d16384f0eb2cbe4d320649370]) successfully deleted
I20260808 11:44:34.308023 22832 catalog_manager.cc:4841] TS 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66:41193): tablet 73194cc434db46b5a20d00731e050b76 (table test-table [id=2ceb469d16384f0eb2cbe4d320649370]) successfully deleted
I20260808 11:44:34.308260 22832 catalog_manager.cc:4841] TS 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381): tablet 73194cc434db46b5a20d00731e050b76 (table test-table [id=2ceb469d16384f0eb2cbe4d320649370]) successfully deleted
I20260808 11:44:34.308771 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 22809
I20260808 11:44:34.376685 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:34.355171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40932 (local address 127.13.158.125:44621)
0808 11:44:34.355373 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:34.355380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:34.358308 (+ 2928us) server_negotiation.cc:407] Connection header received
0808 11:44:34.358615 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:34.358624 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:34.358767 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:34.358857 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:34.362340 (+ 3483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.363499 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.364902 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.365688 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.365824 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:34.370000 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:34.370008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:34.370012 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:34.370067 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:34.373130 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:34.373920 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:34.373933 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:34.373935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:34.374190 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:34.374837 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:34.374844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:34.374850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:34.375174 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:34.375466 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:34.376071 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 11:44:34.376287 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:44:34.378674 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:34.355646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.13.158.124:39083)
0808 11:44:34.355846 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:34.355852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:34.357825 (+ 1973us) server_negotiation.cc:407] Connection header received
0808 11:44:34.358286 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:34.358293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:34.358380 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:34.358456 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:34.361975 (+ 3519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.363580 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.365604 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:34.366226 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:34.366364 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:34.370103 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:34.370109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:34.370114 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:34.370159 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:34.373898 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:34.374920 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:34.374926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:34.374930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:34.375143 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:34.375719 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:34.375724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:34.375727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:34.376202 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:34.376516 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:34.377968 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0808 11:44:34.378232 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:44:35.088438 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.086950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.65:60917)
0808 11:44:35.087835 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.088080 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":515,"threads_started":1}
W20260808 11:44:35.089008 24171 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:45849 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
I20260808 11:44:35.093570 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.092872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.66:55629)
0808 11:44:35.093324 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.093425 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":246,"threads_started":1}
W20260808 11:44:35.093951 24372 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:45849 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
I20260808 11:44:35.096608 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.095772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.67:35267)
0808 11:44:35.096215 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.096359 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":266,"threads_started":1}
W20260808 11:44:35.097090 24576 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:45849 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
I20260808 11:44:35.612352 24671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 560789189ca9482d8310a358e6abfd54)
I20260808 11:44:35.612774 24671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } }
I20260808 11:44:35.613517 24673 raft_consensus.cc:482] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 560789189ca9482d8310a358e6abfd54)
I20260808 11:44:35.613814 24673 raft_consensus.cc:504] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } }
I20260808 11:44:35.614722 24671 leader_election.cc:288] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 560789189ca9482d8310a358e6abfd54 (127.13.158.126:45849), e40f3063557944ce8a011945c0f3c76f (127.13.158.125:44621)
I20260808 11:44:35.617022 24673 leader_election.cc:288] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 560789189ca9482d8310a358e6abfd54 (127.13.158.126:45849), 365c520c6f164df1a8cf66562b8bea8c (127.13.158.124:39083)
I20260808 11:44:35.617182 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.614831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54766)
0808 11:44:35.615938 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.616508 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":902,"thread_start_us":1270,"threads_started":1}
W20260808 11:44:35.620463 23914 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
I20260808 11:44:35.620747 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.616988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54776)
0808 11:44:35.619562 (+ 2574us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.619862 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2391,"threads_started":1}
W20260808 11:44:35.625164 23651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
I20260808 11:44:35.628878 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.628098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54800)
0808 11:44:35.628506 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.628621 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260808 11:44:35.629104 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.615027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44621 (local address 127.0.0.1:40936)
0808 11:44:35.617375 (+ 2348us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.617397 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:35.618009 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:35.618738 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:35.618743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:35.618767 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:35.619418 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.619435 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.622313 (+ 2878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.622319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.625312 (+ 2993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.625332 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.627952 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.627958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.628181 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.628225 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 11:44:35.628318 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 11:44:35.628463 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1935,"thread_start_us":501,"threads_started":1}
W20260808 11:44:35.629164 23914 leader_election.cc:334] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 560789189ca9482d8310a358e6abfd54 (127.13.158.126:45849): Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
I20260808 11:44:35.630249 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.615026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40936 (local address 127.13.158.125:44621)
0808 11:44:35.617718 (+ 2692us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:35.617724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:35.617741 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:44:35.618249 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:35.618255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:35.618330 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:35.618431 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:35.619810 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.622078 (+ 2268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.625552 (+ 3474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.627559 (+ 2007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.627742 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.627824 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:35.629387 (+ 1563us) server_negotiation.cc:299] Negotiation successful
0808 11:44:35.629593 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":2523,"thread_start_us":1501,"threads_started":1}
I20260808 11:44:35.631054 23687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "365c520c6f164df1a8cf66562b8bea8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e40f3063557944ce8a011945c0f3c76f" is_pre_election: true
I20260808 11:44:35.631539 23687 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365c520c6f164df1a8cf66562b8bea8c in term 1.
I20260808 11:44:35.632217 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.630806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54814)
0808 11:44:35.631062 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.631459 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 11:44:35.632294 23915 leader_election.cc:302] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 365c520c6f164df1a8cf66562b8bea8c, e40f3063557944ce8a011945c0f3c76f; no voters: 560789189ca9482d8310a358e6abfd54
W20260808 11:44:35.632650 23651 leader_election.cc:334] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 560789189ca9482d8310a358e6abfd54 (127.13.158.126:45849): Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
I20260808 11:44:35.633230 24671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 11:44:35.633488 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.618528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.13.158.124:39083)
0808 11:44:35.620383 (+ 1855us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:35.620404 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:35.621227 (+ 823us) server_negotiation.cc:407] Connection header received
0808 11:44:35.621287 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:35.621292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:35.621377 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:35.621470 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:35.622547 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.624194 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.629422 (+ 5228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.631230 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.631441 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.631516 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:35.632493 (+ 977us) server_negotiation.cc:299] Negotiation successful
0808 11:44:35.632675 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1661,"thread_start_us":1440,"threads_started":1}
I20260808 11:44:35.633488 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.618005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:39083 (local address 127.0.0.1:43724)
0808 11:44:35.620962 (+ 2957us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.620986 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:35.621117 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:35.621841 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:35.621846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:35.621871 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:35.622310 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.622327 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.624819 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.624825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.629017 (+ 4192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.629044 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.631549 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.631559 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.632200 (+ 641us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.632256 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 11:44:35.632366 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 11:44:35.632546 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2844,"mutex_wait_us":1347,"thread_start_us":1465,"threads_started":1}
I20260808 11:44:35.633561 24671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 560789189ca9482d8310a358e6abfd54)
I20260808 11:44:35.633790 24671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:44:35.634282 23946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e40f3063557944ce8a011945c0f3c76f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "365c520c6f164df1a8cf66562b8bea8c" is_pre_election: true
I20260808 11:44:35.636914 24671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } }
I20260808 11:44:35.637558 23946 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e40f3063557944ce8a011945c0f3c76f in current term 2: Already voted for candidate 365c520c6f164df1a8cf66562b8bea8c in this term.
I20260808 11:44:35.637979 24671 leader_election.cc:288] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [CANDIDATE]: Term 2 election: Requested vote from peers 560789189ca9482d8310a358e6abfd54 (127.13.158.126:45849), e40f3063557944ce8a011945c0f3c76f (127.13.158.125:44621)
I20260808 11:44:35.638589 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.637920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54818)
0808 11:44:35.638194 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.638318 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 11:44:35.638696 23687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "365c520c6f164df1a8cf66562b8bea8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e40f3063557944ce8a011945c0f3c76f"
I20260808 11:44:35.639117 23687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:44:35.639183 23649 leader_election.cc:302] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e40f3063557944ce8a011945c0f3c76f; no voters: 365c520c6f164df1a8cf66562b8bea8c, 560789189ca9482d8310a358e6abfd54
I20260808 11:44:35.639885 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.639349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54828)
0808 11:44:35.639539 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.639707 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260808 11:44:35.640210 23914 leader_election.cc:334] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 560789189ca9482d8310a358e6abfd54 (127.13.158.126:45849): Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
I20260808 11:44:35.642118 23687 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365c520c6f164df1a8cf66562b8bea8c in term 2.
I20260808 11:44:35.642333 24673 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:44:35.642685 23915 leader_election.cc:302] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 365c520c6f164df1a8cf66562b8bea8c, e40f3063557944ce8a011945c0f3c76f; no voters: 560789189ca9482d8310a358e6abfd54
I20260808 11:44:35.643116 24671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:44:35.643790 24671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [term 2 LEADER]: Becoming Leader. State: Replica: 365c520c6f164df1a8cf66562b8bea8c, State: Running, Role: LEADER
I20260808 11:44:35.644374 24671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } }
I20260808 11:44:35.647233 24689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365c520c6f164df1a8cf66562b8bea8c. Latest consensus state: current_term: 2 leader_uuid: "365c520c6f164df1a8cf66562b8bea8c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:35.647859 24689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:35.648785 24691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:35.653855 24691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:35.655172 24691 catalog_manager.cc:1259] Loaded cluster ID: d7473c94efbd4a0085f2b819a32e48a1
I20260808 11:44:35.655275 24691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:35.656987 24691 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:35.658245 24691 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Loaded TSK: 0
I20260808 11:44:35.659245 24691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:44:35.680858 23932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:44:35.691159 23932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:35.696815 23687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [term 2 FOLLOWER]: Refusing update from remote peer 365c520c6f164df1a8cf66562b8bea8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 11:44:35.696939 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.696486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54836)
0808 11:44:35.696664 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.696752 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 11:44:35.698450 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.697918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54840)
0808 11:44:35.698135 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.698202 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 11:44:35.698060 24689 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260808 11:44:35.698679 23914 consensus_peers.cc:487] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c -> Peer 560789189ca9482d8310a358e6abfd54 (127.13.158.126:45849): Couldn't send request to peer 560789189ca9482d8310a358e6abfd54. Status: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 11:44:35.702203 24673 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365c520c6f164df1a8cf66562b8bea8c. Latest consensus state: current_term: 2 leader_uuid: "365c520c6f164df1a8cf66562b8bea8c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:35.702658 24673 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:44:35.704592 24671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "365c520c6f164df1a8cf66562b8bea8c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:35.705165 24671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:35.705855 24673 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "365c520c6f164df1a8cf66562b8bea8c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "560789189ca9482d8310a358e6abfd54" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45849 } } peers { permanent_uuid: "e40f3063557944ce8a011945c0f3c76f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44621 } attrs { promote: false } } peers { permanent_uuid: "365c520c6f164df1a8cf66562b8bea8c" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 39083 } attrs { promote: false } } }
I20260808 11:44:35.706277 24673 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40f3063557944ce8a011945c0f3c76f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:44:35.742487 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.715346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:40381 (local address 127.0.0.1:46676)
0808 11:44:35.715730 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.715764 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:35.715899 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:35.729322 (+ 13423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:35.729330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:35.729365 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:35.730059 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.730076 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.732252 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.732261 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.735687 (+ 3426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.735708 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.741408 (+ 5700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.741416 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.741649 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.741693 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 11:44:35.741808 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 11:44:35.742003 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230}
I20260808 11:44:35.742527 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.716192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.13.158.67:40381)
0808 11:44:35.728780 (+ 12588us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:35.728789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:35.728813 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:44:35.728895 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:35.728904 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:35.729053 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:35.730230 (+ 1177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:35.730468 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.732084 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.735964 (+ 3880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.737754 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.741158 (+ 3404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.741330 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:35.741870 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 11:44:35.742033 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12439,"thread_start_us":12949,"threads_started":1}
I20260808 11:44:35.743633 24483 tablet_service.cc:1467] Processing CreateTablet for tablet 990ce76beb4f4069b3cb9c7f551fb2a6 (DEFAULT_TABLE table=test-table [id=59f62435848b4f7c806f17b9a3136437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:35.744165 24483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990ce76beb4f4069b3cb9c7f551fb2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:35.747792 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.714569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41193 (local address 127.0.0.1:47782)
0808 11:44:35.714885 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.714905 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:35.715012 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:35.738786 (+ 23774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:35.738791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:35.738832 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:35.739076 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.739090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.741474 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.741481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.744154 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.744170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.746902 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.746908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.747144 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.747195 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 11:44:35.747280 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 11:44:35.747404 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":377}
I20260808 11:44:35.748778 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.715467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.13.158.66:41193)
0808 11:44:35.738299 (+ 22832us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:35.738317 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:35.738337 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:44:35.738443 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:35.738447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:35.738571 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:35.739236 (+ 665us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:35.739435 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.740900 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.745135 (+ 4235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.746755 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.746893 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.747050 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:35.748062 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0808 11:44:35.748233 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22699,"thread_start_us":22586,"threads_started":1}
I20260808 11:44:35.749464 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.715540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:38839 (local address 127.0.0.1:47892)
0808 11:44:35.739023 (+ 23483us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.739045 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:35.739220 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:35.739803 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:35.739811 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:35.739849 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:35.740770 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.740783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.742493 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.742498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.746333 (+ 3835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.746348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.748527 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.748534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.748784 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.748821 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 11:44:35.748910 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 11:44:35.749044 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23281,"mutex_wait_us":470,"thread_start_us":23799,"threads_started":1}
I20260808 11:44:35.749684 24282 tablet_service.cc:1467] Processing CreateTablet for tablet 990ce76beb4f4069b3cb9c7f551fb2a6 (DEFAULT_TABLE table=test-table [id=59f62435848b4f7c806f17b9a3136437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:35.750257 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.716170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47892 (local address 127.13.158.65:38839)
0808 11:44:35.738129 (+ 21959us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:35.738136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:35.739151 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 11:44:35.739410 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:35.739419 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:35.739592 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:35.739992 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:35.740970 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.742257 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.746502 (+ 4245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.747947 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.748105 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.748261 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:35.749683 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0808 11:44:35.749831 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21793,"thread_start_us":21971,"threads_started":1}
I20260808 11:44:35.750273 24282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990ce76beb4f4069b3cb9c7f551fb2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:35.751597 24084 tablet_service.cc:1467] Processing CreateTablet for tablet 990ce76beb4f4069b3cb9c7f551fb2a6 (DEFAULT_TABLE table=test-table [id=59f62435848b4f7c806f17b9a3136437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:35.752126 24084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 990ce76beb4f4069b3cb9c7f551fb2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:35.755594 24707 tablet_bootstrap.cc:492] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: Bootstrap starting.
I20260808 11:44:35.758260 24707 tablet_bootstrap.cc:654] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:35.758538 24709 tablet_bootstrap.cc:492] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: Bootstrap starting.
I20260808 11:44:35.760694 24710 tablet_bootstrap.cc:492] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: Bootstrap starting.
I20260808 11:44:35.762398 24709 tablet_bootstrap.cc:654] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:35.763056 24707 tablet_bootstrap.cc:492] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: No bootstrap required, opened a new log
I20260808 11:44:35.763266 24707 ts_tablet_manager.cc:1397] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260808 11:44:35.764170 24710 tablet_bootstrap.cc:654] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:35.764649 24707 raft_consensus.cc:348] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.765148 24707 raft_consensus.cc:374] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:35.765372 24707 raft_consensus.cc:729] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964f9eaed9c843cd8ef86ee7c526a783, State: Initialized, Role: FOLLOWER
I20260808 11:44:35.765429 24709 tablet_bootstrap.cc:492] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: No bootstrap required, opened a new log
I20260808 11:44:35.765594 24709 ts_tablet_manager.cc:1397] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260808 11:44:35.765753 24707 consensus_queue.cc:260] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.766808 24709 raft_consensus.cc:348] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.767148 24709 raft_consensus.cc:374] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:35.767313 24709 raft_consensus.cc:729] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff970853d2f47bdb3eecc00a9310ec4, State: Initialized, Role: FOLLOWER
I20260808 11:44:35.767709 24709 consensus_queue.cc:260] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.768394 24707 ts_tablet_manager.cc:1428] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260808 11:44:35.768934 24709 ts_tablet_manager.cc:1428] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 11:44:35.771703 24575 heartbeater.cc:503] Master 127.13.158.124:39083 was elected leader, sending a full tablet report...
I20260808 11:44:35.772213 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.770413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.66:44713)
0808 11:44:35.771474 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.771708 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":844,"threads_started":1}
I20260808 11:44:35.772365 24371 heartbeater.cc:503] Master 127.13.158.124:39083 was elected leader, sending a full tablet report...
I20260808 11:44:35.775426 24710 tablet_bootstrap.cc:492] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: No bootstrap required, opened a new log
I20260808 11:44:35.775658 24710 ts_tablet_manager.cc:1397] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.006s
I20260808 11:44:35.776724 24710 raft_consensus.cc:348] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.776971 24710 raft_consensus.cc:374] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:35.777143 24710 raft_consensus.cc:729] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f196183a33d49499432ed6622045abf, State: Initialized, Role: FOLLOWER
I20260808 11:44:35.777468 24710 consensus_queue.cc:260] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.778757 24710 ts_tablet_manager.cc:1428] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:35.781029 24170 heartbeater.cc:503] Master 127.13.158.124:39083 was elected leader, sending a full tablet report...
I20260808 11:44:35.781097 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.773152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.66:45377)
0808 11:44:35.774187 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.774270 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":929}
I20260808 11:44:35.782858 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.767308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.67:39203)
0808 11:44:35.782051 (+ 14743us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.782307 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14613,"thread_start_us":14388,"threads_started":1}
I20260808 11:44:35.783973 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.783559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.67:51337)
0808 11:44:35.783741 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.783822 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 11:44:35.785070 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.779212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.65:50175)
0808 11:44:35.784491 (+ 5279us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.784648 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5147,"thread_start_us":4953,"threads_started":1}
I20260808 11:44:35.785974 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.785644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.65:37009)
0808 11:44:35.785769 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.785830 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 11:44:35.869115 24377 tablet.cc:2369] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:44:35.934477 24179 tablet.cc:2369] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:44:35.957505 24718 raft_consensus.cc:482] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:35.957823 24718 raft_consensus.cc:504] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.960052 24718 leader_election.cc:288] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381), 3f196183a33d49499432ed6622045abf (127.13.158.65:38839)
I20260808 11:44:35.969434 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.960875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:33101 (local address 127.13.158.67:40381)
0808 11:44:35.961116 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:35.961124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:35.961779 (+ 655us) server_negotiation.cc:407] Connection header received
0808 11:44:35.961992 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:35.961998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:35.962067 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:35.962124 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:35.963304 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.964497 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.966873 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.968044 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.968317 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.968415 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:35.968856 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 11:44:35.969001 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 11:44:35.969655 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.960299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:40381 (local address 127.13.158.66:33101)
0808 11:44:35.961667 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.961684 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:35.961814 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:35.962288 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:35.962292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:35.962319 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:35.963146 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.963159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.964603 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.964608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.966674 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.966687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.968336 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.968343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.968657 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.968696 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 11:44:35.968797 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 11:44:35.968977 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1227,"mutex_wait_us":1,"thread_start_us":923,"threads_started":1}
I20260808 11:44:35.970604 24510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990ce76beb4f4069b3cb9c7f551fb2a6" candidate_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964f9eaed9c843cd8ef86ee7c526a783" is_pre_election: true
I20260808 11:44:35.970844 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.960578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:42435 (local address 127.13.158.65:38839)
0808 11:44:35.960833 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:35.960850 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:35.961429 (+ 579us) server_negotiation.cc:407] Connection header received
0808 11:44:35.961626 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:35.961630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:35.961690 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:35.961793 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:35.962957 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.964372 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.967600 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.968929 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.969072 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.969157 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:35.970390 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0808 11:44:35.970533 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:44:35.970953 24510 raft_consensus.cc:2454] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff970853d2f47bdb3eecc00a9310ec4 in term 0.
I20260808 11:44:35.971372 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.960256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:38839 (local address 127.13.158.66:42435)
0808 11:44:35.960587 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.961194 (+ 607us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:35.961435 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:35.961859 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:35.961864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:35.961940 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:35.962765 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.962786 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.964513 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.964519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.967417 (+ 2898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:35.967432 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:35.969982 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:35.969992 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:35.970195 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:35.970225 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:44:35.970347 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 11:44:35.970500 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":56}
I20260808 11:44:35.971670 24221 leader_election.cc:302] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff970853d2f47bdb3eecc00a9310ec4, 964f9eaed9c843cd8ef86ee7c526a783; no voters:
I20260808 11:44:35.972177 24718 raft_consensus.cc:2781] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:35.972339 24718 raft_consensus.cc:482] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:35.972458 24718 raft_consensus.cc:3037] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:35.972648 24116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990ce76beb4f4069b3cb9c7f551fb2a6" candidate_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f196183a33d49499432ed6622045abf" is_pre_election: true
I20260808 11:44:35.973380 24116 raft_consensus.cc:2454] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff970853d2f47bdb3eecc00a9310ec4 in term 0.
I20260808 11:44:35.976244 24718 raft_consensus.cc:504] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.977164 24718 leader_election.cc:288] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381), 3f196183a33d49499432ed6622045abf (127.13.158.65:38839)
I20260808 11:44:35.977717 24510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990ce76beb4f4069b3cb9c7f551fb2a6" candidate_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964f9eaed9c843cd8ef86ee7c526a783"
I20260808 11:44:35.977972 24510 raft_consensus.cc:3037] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:35.977969 24116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "990ce76beb4f4069b3cb9c7f551fb2a6" candidate_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f196183a33d49499432ed6622045abf"
I20260808 11:44:35.978269 24116 raft_consensus.cc:3037] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:35.980545 24510 raft_consensus.cc:2454] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff970853d2f47bdb3eecc00a9310ec4 in term 1.
I20260808 11:44:35.980973 24221 leader_election.cc:302] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff970853d2f47bdb3eecc00a9310ec4, 964f9eaed9c843cd8ef86ee7c526a783; no voters:
I20260808 11:44:35.981405 24718 raft_consensus.cc:2781] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:35.981951 24116 raft_consensus.cc:2454] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff970853d2f47bdb3eecc00a9310ec4 in term 1.
I20260808 11:44:35.982295 24718 raft_consensus.cc:686] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff970853d2f47bdb3eecc00a9310ec4, State: Running, Role: LEADER
I20260808 11:44:35.982909 24718 consensus_queue.cc:237] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } }
I20260808 11:44:35.983526 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:35.982975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.66:56799)
0808 11:44:35.983169 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:35.983342 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 11:44:35.984128 24579 tablet.cc:2369] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:44:35.987367 23932 catalog_manager.cc:5495] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66). New cstate: current_term: 1 leader_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff970853d2f47bdb3eecc00a9310ec4" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 } health_report { overall_health: UNKNOWN } } }
I20260808 11:44:36.054652 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.048496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47794 (local address 127.13.158.66:41193)
0808 11:44:36.048698 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:36.048702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:36.048731 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:44:36.048994 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:36.048998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:36.049077 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:36.049201 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:36.049620 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:36.050494 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:36.052251 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:36.052875 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:36.053017 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:36.053467 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:36.053999 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:36.054152 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:36.054253 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 11:44:36.054332 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:44:36.060655 24510 raft_consensus.cc:1261] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 1 FOLLOWER]: Refusing update from remote peer 2ff970853d2f47bdb3eecc00a9310ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:36.061228 24116 raft_consensus.cc:1261] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 1 FOLLOWER]: Refusing update from remote peer 2ff970853d2f47bdb3eecc00a9310ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:36.061510 24723 consensus_queue.cc:1035] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964f9eaed9c843cd8ef86ee7c526a783" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 40381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:36.062095 24718 consensus_queue.cc:1035] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f196183a33d49499432ed6622045abf" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:36.065176 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.064484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.67:52093)
0808 11:44:36.064688 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.064859 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 11:44:36.065218 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.064475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.65:52503)
0808 11:44:36.064736 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.064872 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260808 11:44:36.070986 24726 mvcc.cc:205] Tried to move back new op lower bound from 7316232093935636480 to 7316232093631844352. Current Snapshot: MvccSnapshot[applied={T|T < 7316232093935636480}]
I20260808 11:44:36.073525 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.072926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.65:42115)
0808 11:44:36.073226 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.073321 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260808 11:44:36.073985 24727 mvcc.cc:205] Tried to move back new op lower bound from 7316232093935636480 to 7316232093631844352. Current Snapshot: MvccSnapshot[applied={T|T < 7316232093935636480}]
I20260808 11:44:36.074396 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.073881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.66:34793)
0808 11:44:36.074129 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.074224 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 11:44:36.075057 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.074331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.13.158.67:33097)
0808 11:44:36.074757 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.074840 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294}
I20260808 11:44:36.089929 23932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:36.090302 23932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:36.096607 23932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 365c520c6f164df1a8cf66562b8bea8c: Sending DeleteTablet for 3 replicas of tablet 990ce76beb4f4069b3cb9c7f551fb2a6
I20260808 11:44:36.097812 24282 tablet_service.cc:1514] Processing DeleteTablet for tablet 990ce76beb4f4069b3cb9c7f551fb2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:36 UTC) from {username='oryx', principal='oryx/127.13.158.124@KRBTEST.COM'} at 127.0.0.1:47782
I20260808 11:44:36.097892 24483 tablet_service.cc:1514] Processing DeleteTablet for tablet 990ce76beb4f4069b3cb9c7f551fb2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:36 UTC) from {username='oryx', principal='oryx/127.13.158.124@KRBTEST.COM'} at 127.0.0.1:46676
I20260808 11:44:36.098541 24743 tablet_replica.cc:330] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: stopping tablet replica
I20260808 11:44:36.098757 24743 raft_consensus.cc:2229] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:36.099102 24743 raft_consensus.cc:2258] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:36.099157 24084 tablet_service.cc:1514] Processing DeleteTablet for tablet 990ce76beb4f4069b3cb9c7f551fb2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:36 UTC) from {username='oryx', principal='oryx/127.13.158.124@KRBTEST.COM'} at 127.0.0.1:47892
I20260808 11:44:36.100630 24743 ts_tablet_manager.cc:1905] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:36.104306 24744 tablet_replica.cc:330] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: stopping tablet replica
I20260808 11:44:36.104508 24744 raft_consensus.cc:2229] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:44:36.105225 24744 raft_consensus.cc:2258] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:36.105286 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 23903
I20260808 11:44:36.111833 24744 ts_tablet_manager.cc:1905] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:36.115626 24743 ts_tablet_manager.cc:1918] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:36.115772 24743 log.cc:1196] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/990ce76beb4f4069b3cb9c7f551fb2a6
I20260808 11:44:36.116293 24743 ts_tablet_manager.cc:1939] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 964f9eaed9c843cd8ef86ee7c526a783: Deleting consensus metadata
I20260808 11:44:36.117578 23916 catalog_manager.cc:4841] TS 964f9eaed9c843cd8ef86ee7c526a783 (127.13.158.67:40381): tablet 990ce76beb4f4069b3cb9c7f551fb2a6 (table test-table [id=59f62435848b4f7c806f17b9a3136437]) successfully deleted
I20260808 11:44:36.121063 24745 tablet_replica.cc:330] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: stopping tablet replica
I20260808 11:44:36.121276 24745 raft_consensus.cc:2229] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:36.121553 24745 raft_consensus.cc:2258] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:36.121786 24744 ts_tablet_manager.cc:1918] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:36.121986 24744 log.cc:1196] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/990ce76beb4f4069b3cb9c7f551fb2a6
I20260808 11:44:36.122653 24744 ts_tablet_manager.cc:1939] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 2ff970853d2f47bdb3eecc00a9310ec4: Deleting consensus metadata
I20260808 11:44:36.122932 24745 ts_tablet_manager.cc:1905] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:36.123890 23916 catalog_manager.cc:4841] TS 2ff970853d2f47bdb3eecc00a9310ec4 (127.13.158.66:41193): tablet 990ce76beb4f4069b3cb9c7f551fb2a6 (table test-table [id=59f62435848b4f7c806f17b9a3136437]) successfully deleted
I20260808 11:44:36.130012 24745 ts_tablet_manager.cc:1918] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:36.130208 24745 log.cc:1196] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/990ce76beb4f4069b3cb9c7f551fb2a6
I20260808 11:44:36.130829 24745 ts_tablet_manager.cc:1939] T 990ce76beb4f4069b3cb9c7f551fb2a6 P 3f196183a33d49499432ed6622045abf: Deleting consensus metadata
I20260808 11:44:36.131930 23915 catalog_manager.cc:4841] TS 3f196183a33d49499432ed6622045abf (127.13.158.65:38839): tablet 990ce76beb4f4069b3cb9c7f551fb2a6 (table test-table [id=59f62435848b4f7c806f17b9a3136437]) successfully deleted
I20260808 11:44:36.137027 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.106906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47904 (local address 127.13.158.65:38839)
0808 11:44:36.107084 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:36.107090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:36.120738 (+ 13648us) server_negotiation.cc:407] Connection header received
0808 11:44:36.120937 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:36.120942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:36.121026 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:36.121169 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:36.123074 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:36.124219 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:36.125959 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:36.126632 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:36.126772 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:36.130283 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:36.130299 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:36.130319 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:36.130358 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:36.134320 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:36.135013 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:36.135021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:36.135023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:36.135139 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:36.135519 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:36.135530 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:36.135532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:36.135873 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:36.136169 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:36.136466 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 11:44:36.136682 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:44:36.137674 24165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:36.183386 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.182879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54848)
0808 11:44:36.183083 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.183197 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 11:44:36.184578 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.184189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54858)
0808 11:44:36.184329 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.184424 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 11:44:36.261436 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 23903
I20260808 11:44:36.297084 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 24200
I20260808 11:44:36.312670 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.297860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.13.158.66:41193)
0808 11:44:36.298062 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:36.298069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:36.298226 (+ 157us) server_negotiation.cc:407] Connection header received
0808 11:44:36.298447 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:36.298453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:36.298528 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:36.298672 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:36.300271 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:36.300984 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:36.302794 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:36.303213 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:36.303308 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:36.306627 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:36.306640 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:36.306653 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:36.306683 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:36.309816 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:36.310560 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:36.310566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:36.310568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:36.310829 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:36.311275 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:36.311278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:36.311280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:36.311607 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:36.311851 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:36.312227 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 11:44:36.312389 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 11:44:36.313366 24361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:36.432283 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 24200
I20260808 11:44:36.463282 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 24395
I20260808 11:44:36.477114 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.464101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.13.158.67:40381)
0808 11:44:36.464300 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:36.464305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:36.464323 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:44:36.464471 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:36.464476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:36.464539 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:36.464869 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:36.466232 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:36.467076 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:36.468287 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:36.468664 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:36.468757 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:36.471293 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:36.471307 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:36.471325 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:36.471356 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:36.474281 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:36.474959 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:36.474966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:36.474968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:36.475088 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:36.475469 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:36.475481 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:36.475484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:36.475944 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:36.476272 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:36.476574 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 11:44:36.476811 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:44:36.477778 24567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:36.569407 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 24395
I20260808 11:44:36.595731 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 23476
I20260808 11:44:36.597074 23711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:36.714473 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 23476
I20260808 11:44:36.721704 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.720662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54872)
0808 11:44:36.721447 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.721543 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":560,"threads_started":1}
I20260808 11:44:36.723054 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:36.722683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45849 (local address 127.0.0.1:54878)
0808 11:44:36.722876 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:36.722944 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:45849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 11:44:36.743695 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 23615
I20260808 11:44:36.744664 23970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:36.865307 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 23615
2026-08-08T11:44:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10333 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:36 dist-test-slave-39l8 krb5kdc[24757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:36 dist-test-slave-39l8 krb5kdc[24757](info): set up 2 sockets
Aug 08 11:44:36 dist-test-slave-39l8 krb5kdc[24757](info): commencing operation
krb5kdc: starting...
W20260808 11:44:38.165297 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:38 dist-test-slave-39l8 krb5kdc[24757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.126.keytab.
Entry for principal oryx/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.126.keytab.
I20260808 11:44:38.394196 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:46771
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.126.keytab
--principal=oryx/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:46771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:38.722559 24777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:38.722873 24777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:38.723063 24777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:38.729866 24777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:38.729957 24777 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 11:44:38.729997 24777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:38.730113 24777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:38.730194 24777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:38.730299 24777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:38.739763 24777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:46771
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.126.keytab
--principal=oryx/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:46771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:38.741007 24777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:38.742314 24777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:38.753829 24787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:38.753760 24789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:38.753823 24786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:38.754912 24777 server_base.cc:1034] running on GCE node
I20260808 11:44:38.755592 24777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:38.756856 24777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:38.758101 24777 hybrid_clock.cc:648] HybridClock initialized: now 1786189478758037 us; error 55 us; skew 500 ppm
I20260808 11:44:38.758426 24777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:38 dist-test-slave-39l8 krb5kdc[24757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189478, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:38.761425 24777 init.cc:377] Logged in from keytab as oryx/127.13.158.126@KRBTEST.COM (short username oryx)
I20260808 11:44:38.762897 24777 webserver.cc:460] Webserver started at http://127.13.158.126:43659/ using document root <none> and password file <none>
I20260808 11:44:38.763593 24777 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:38.763741 24777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:38.764076 24777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:38.766979 24777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "2a4b001fae7e4a1b8cf763a9cf0685db"
format_stamp: "Formatted at 2026-08-08 11:44:38 on dist-test-slave-39l8"
server_key: "7a7fcedc102f0b768f52791be954c085"
server_key_iv: "e8174ad8fe6f7a6d60ad8516d5d42abe"
server_key_version: "encryptionkey@0"
I20260808 11:44:38.767767 24777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "2a4b001fae7e4a1b8cf763a9cf0685db"
format_stamp: "Formatted at 2026-08-08 11:44:38 on dist-test-slave-39l8"
server_key: "7a7fcedc102f0b768f52791be954c085"
server_key_iv: "e8174ad8fe6f7a6d60ad8516d5d42abe"
server_key_version: "encryptionkey@0"
I20260808 11:44:38.772692 24777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:44:38.775986 24800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:38.777288 24777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:44:38.777477 24777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "2a4b001fae7e4a1b8cf763a9cf0685db"
format_stamp: "Formatted at 2026-08-08 11:44:38 on dist-test-slave-39l8"
server_key: "7a7fcedc102f0b768f52791be954c085"
server_key_iv: "e8174ad8fe6f7a6d60ad8516d5d42abe"
server_key_version: "encryptionkey@0"
I20260808 11:44:38.777699 24777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:38.794394 24777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:38.798317 24777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:38.798588 24777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:38.821130 24777 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:46771
I20260808 11:44:38.821151 24893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:46771 every 8 connection(s)
I20260808 11:44:38.822459 24777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:38.825089 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 24777
I20260808 11:44:38.825316 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:38.825968 13945 external_mini_cluster.cc:1428] Setting key 5055e4f63a05215ca5785331c37eeaaf
I20260808 11:44:38.826390 24894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:38.840546 24894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: Bootstrap starting.
Aug 08 11:44:38 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:38.843658 24894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:38.844717 24894 log.cc:824] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:38.847553 24894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: No bootstrap required, opened a new log
I20260808 11:44:38.849270 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:38.827910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.13.158.126:46771)
0808 11:44:38.828660 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:38.828671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:38.828712 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:44:38.830942 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:38.830963 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:38.831317 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:38.831616 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:38.833422 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:38.834674 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:38.836482 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:38.837181 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:38.837316 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:38.842046 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:38.842054 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:38.842070 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:38.842103 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:38.845220 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:38.845847 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:38.845859 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:38.845864 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:38.846098 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:38.846496 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:38.846501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:38.846506 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:38.847089 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:38.847417 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:38.847888 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 11:44:38.848337 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":367,"threads_started":1}
I20260808 11:44:38.855479 24894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b001fae7e4a1b8cf763a9cf0685db" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46771 } }
I20260808 11:44:38.855804 24894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:38.855903 24894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4b001fae7e4a1b8cf763a9cf0685db, State: Initialized, Role: FOLLOWER
I20260808 11:44:38.856418 24894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b001fae7e4a1b8cf763a9cf0685db" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46771 } }
I20260808 11:44:38.856628 24894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:38.856771 24894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:38.856990 24894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:38.859534 24894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b001fae7e4a1b8cf763a9cf0685db" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46771 } }
I20260808 11:44:38.860101 24894 leader_election.cc:302] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a4b001fae7e4a1b8cf763a9cf0685db; no voters:
I20260808 11:44:38.860667 24894 leader_election.cc:288] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:38.860792 24901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:38.861502 24901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [term 1 LEADER]: Becoming Leader. State: Replica: 2a4b001fae7e4a1b8cf763a9cf0685db, State: Running, Role: LEADER
I20260808 11:44:38.862098 24901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b001fae7e4a1b8cf763a9cf0685db" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46771 } }
I20260808 11:44:38.863042 24894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:38.865163 24905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a4b001fae7e4a1b8cf763a9cf0685db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b001fae7e4a1b8cf763a9cf0685db" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46771 } } }
I20260808 11:44:38.865319 24906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a4b001fae7e4a1b8cf763a9cf0685db. Latest consensus state: current_term: 1 leader_uuid: "2a4b001fae7e4a1b8cf763a9cf0685db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4b001fae7e4a1b8cf763a9cf0685db" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 46771 } } }
I20260808 11:44:38.865567 24905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:38.865581 24906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:38.868089 24915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:38.873684 24915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:38.882825 24915 catalog_manager.cc:1347] Generated new cluster ID: cf92c1aadbc44ad8b516297d81a82ddd
I20260808 11:44:38.883016 24915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:38.900315 24915 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:38.902222 24915 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:38.909639 24915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: Generated new TSK 0
I20260808 11:44:38.910532 24915 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab.
Entry for principal oryx/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab.
I20260808 11:44:38.968602 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46771
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab
--principal=oryx/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:39.297650 24937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:39.297917 24937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:39.298045 24937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:39.305065 24937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:39.305162 24937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:39.305341 24937 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:44:39.305469 24937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:39.314976 24937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab
--principal=oryx/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:46771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:39.316185 24937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:39.317557 24937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:39.330576 24943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:39.330613 24948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:39.330734 24946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:39.331748 24937 server_base.cc:1034] running on GCE node
I20260808 11:44:39.332324 24937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:39.333209 24937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:39.334417 24937 hybrid_clock.cc:648] HybridClock initialized: now 1786189479334342 us; error 49 us; skew 500 ppm
I20260808 11:44:39.334683 24937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:39 dist-test-slave-39l8 krb5kdc[24757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189479, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:39.337744 24937 init.cc:377] Logged in from keytab as oryx/127.13.158.65@KRBTEST.COM (short username oryx)
I20260808 11:44:39.339110 24937 webserver.cc:460] Webserver started at http://127.13.158.65:35861/ using document root <none> and password file <none>
I20260808 11:44:39.339726 24937 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:39.339839 24937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:39.340096 24937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:39.342826 24937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "85290b6a4c444b2f83e0e91d67a7ac2a"
format_stamp: "Formatted at 2026-08-08 11:44:39 on dist-test-slave-39l8"
server_key: "8a7679db19ab3842ca72edc77c9327d3"
server_key_iv: "b543ed9d7717d3e9817faf92992b450c"
server_key_version: "encryptionkey@0"
I20260808 11:44:39.343617 24937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "85290b6a4c444b2f83e0e91d67a7ac2a"
format_stamp: "Formatted at 2026-08-08 11:44:39 on dist-test-slave-39l8"
server_key: "8a7679db19ab3842ca72edc77c9327d3"
server_key_iv: "b543ed9d7717d3e9817faf92992b450c"
server_key_version: "encryptionkey@0"
I20260808 11:44:39.348400 24937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:44:39.351706 24956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:39.352957 24937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:44:39.353130 24937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "85290b6a4c444b2f83e0e91d67a7ac2a"
format_stamp: "Formatted at 2026-08-08 11:44:39 on dist-test-slave-39l8"
server_key: "8a7679db19ab3842ca72edc77c9327d3"
server_key_iv: "b543ed9d7717d3e9817faf92992b450c"
server_key_version: "encryptionkey@0"
I20260808 11:44:39.353343 24937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:39.391752 24937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:39.396057 24937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:39.396356 24937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:39.399081 24937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:39.399183 24937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:39.399317 24937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:39.399401 24937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:44:39 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189479, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:39.423118 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:39.399637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:49387 (local address 127.13.158.126:46771)
0808 11:44:39.400403 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:39.400408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:39.401268 (+ 860us) server_negotiation.cc:407] Connection header received
0808 11:44:39.403465 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:39.403471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:39.403563 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:39.403683 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:39.405805 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.407151 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.410506 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.411022 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.411176 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:39.415837 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:39.415842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:39.415845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:39.415891 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:39.419401 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:39.420550 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:39.420554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:39.420558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:39.420679 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:39.421302 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:39.421308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:39.421314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:39.421636 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:39.421810 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:39.422482 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 11:44:39.422716 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":526,"threads_started":1}
I20260808 11:44:39.424503 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:39.400280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46771 (local address 127.13.158.65:49387)
0808 11:44:39.401107 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:39.401146 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:39.403176 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:39.403933 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:39.403941 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:39.404635 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:39.405481 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:39.405525 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.407377 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.407386 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:39.410307 (+ 2921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:39.410327 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.411303 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.411317 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:39.411613 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:39.412495 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:39.412517 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:39.415575 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:39.419592 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:39.419618 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:39.419632 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:39.420401 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:39.420819 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:39.420827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:39.420837 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:39.421160 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:39.421792 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:39.421799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:39.422137 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 11:44:39.422404 (+ 267us) client_negotiation.cc:240] Negotiation successful
0808 11:44:39.422971 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":388,"threads_started":1}
I20260808 11:44:39.445422 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:39.431066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:58883 (local address 127.13.158.126:46771)
0808 11:44:39.431280 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:39.431287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:39.432451 (+ 1164us) server_negotiation.cc:407] Connection header received
0808 11:44:39.432647 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:39.432650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:39.432712 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:39.432825 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:39.433753 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.434427 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.436246 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.436844 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.436977 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:39.438911 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:39.438916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:39.438919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:39.438948 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:39.442177 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:39.443158 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:39.443165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:39.443170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:39.443339 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:39.443817 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:39.443825 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:39.443829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:39.444146 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:39.444310 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:39.444848 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 11:44:39.445076 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:44:39.445559 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:39.431346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46771 (local address 127.13.158.65:58883)
0808 11:44:39.432322 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:39.432348 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:39.432523 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:39.432857 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:39.432861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:39.433307 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:39.433595 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:39.433606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.434579 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.434590 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:39.436104 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:39.436126 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.437064 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.437076 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:39.437232 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:39.437930 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:39.437946 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:39.438667 (+ 721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:39.442328 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:39.442333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:39.442337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:39.442968 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:39.443490 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:39.443497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:39.443501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:39.443683 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:39.444278 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:39.444285 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:39.444534 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 11:44:39.444704 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 11:44:39.445111 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":536,"threads_started":1}
I20260808 11:44:39.456372 24937 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:44101
I20260808 11:44:39.456408 25113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:44101 every 8 connection(s)
I20260808 11:44:39.457693 24937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:39.459873 25114 heartbeater.cc:347] Connected to a master server at 127.13.158.126:46771
I20260808 11:44:39.460055 25114 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:39.460618 25114 heartbeater.cc:511] Master 127.13.158.126:46771 requested a full tablet report, sending...
I20260808 11:44:39.462368 24823 ts_manager.cc:194] Registered new tserver with Master: 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101)
I20260808 11:44:39.463546 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 24937
I20260808 11:44:39.463762 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:44:39.464066 24823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:49387
I20260808 11:44:39.464527 13945 external_mini_cluster.cc:1428] Setting key a05c53f133811268e058c7ed56b90df9
WARNING: no policy specified for oryx/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab.
Entry for principal oryx/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab.
I20260808 11:44:39.513828 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46771
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab
--principal=oryx/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:39.839013 25118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:39.839298 25118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:39.839444 25118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:39.846403 25118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:39.846516 25118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:39.846621 25118 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:44:39.846787 25118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:39.856360 25118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab
--principal=oryx/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:46771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:39.857571 25118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:39.859012 25118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:39.872779 25125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:39.872998 25124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:39.872805 25128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:39.873485 25118 server_base.cc:1034] running on GCE node
I20260808 11:44:39.874167 25118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:39.875196 25118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:39.876412 25118 hybrid_clock.cc:648] HybridClock initialized: now 1786189479876348 us; error 36 us; skew 500 ppm
I20260808 11:44:39.876703 25118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:39 dist-test-slave-39l8 krb5kdc[24757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189479, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:39.880162 25118 init.cc:377] Logged in from keytab as oryx/127.13.158.66@KRBTEST.COM (short username oryx)
I20260808 11:44:39.881709 25118 webserver.cc:460] Webserver started at http://127.13.158.66:42183/ using document root <none> and password file <none>
I20260808 11:44:39.882462 25118 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:39.882587 25118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:39.882978 25118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:39.885882 25118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "fd4cb1cfaa864c988999beb92695e272"
format_stamp: "Formatted at 2026-08-08 11:44:39 on dist-test-slave-39l8"
server_key: "0f9126146704cf5b568a69892faab886"
server_key_iv: "1275a10e8c7760b9f6e62ec43a55ede9"
server_key_version: "encryptionkey@0"
I20260808 11:44:39.886698 25118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "fd4cb1cfaa864c988999beb92695e272"
format_stamp: "Formatted at 2026-08-08 11:44:39 on dist-test-slave-39l8"
server_key: "0f9126146704cf5b568a69892faab886"
server_key_iv: "1275a10e8c7760b9f6e62ec43a55ede9"
server_key_version: "encryptionkey@0"
I20260808 11:44:39.891697 25118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:44:39.895068 25135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:39.896405 25118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:44:39.896569 25118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "fd4cb1cfaa864c988999beb92695e272"
format_stamp: "Formatted at 2026-08-08 11:44:39 on dist-test-slave-39l8"
server_key: "0f9126146704cf5b568a69892faab886"
server_key_iv: "1275a10e8c7760b9f6e62ec43a55ede9"
server_key_version: "encryptionkey@0"
I20260808 11:44:39.896785 25118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:39.914225 25118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:39.918493 25118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:39.918808 25118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:39.921473 25118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:39.921602 25118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:39.921754 25118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:39.921828 25118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:44:39 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189479, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:39.945073 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:39.921969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:53495 (local address 127.13.158.126:46771)
0808 11:44:39.922173 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:39.922182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:39.923516 (+ 1334us) server_negotiation.cc:407] Connection header received
0808 11:44:39.925564 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:39.925570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:39.925660 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:39.925743 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:39.928192 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.929518 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.932140 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.932742 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.932888 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:39.936978 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:39.936983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:39.936986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:39.937017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:39.940890 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:39.941823 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:39.941827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:39.941829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:39.941969 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:39.942639 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:39.942647 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:39.942651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:39.943078 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:39.943262 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:39.944214 (+ 952us) server_negotiation.cc:299] Negotiation successful
0808 11:44:39.944601 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:44:39.946094 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:39.922482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46771 (local address 127.13.158.66:53495)
0808 11:44:39.923361 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:39.923396 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:39.925214 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:39.926132 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:39.926139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:39.927025 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:39.927958 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:39.927979 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.929732 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.929739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:39.931956 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:39.931968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.933017 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.933028 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:39.933251 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:39.933925 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:39.933946 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:39.936696 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:39.941038 (+ 4342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:39.941052 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:39.941067 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:39.941653 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:39.942103 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:39.942108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:39.942111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:39.942464 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:39.943230 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:39.943242 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:39.943730 (+ 488us) client_negotiation.cc:769] Sending connection context
0808 11:44:39.944059 (+ 329us) client_negotiation.cc:240] Negotiation successful
0808 11:44:39.944682 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":400,"threads_started":1}
I20260808 11:44:39.968510 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:39.951732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:40215 (local address 127.13.158.126:46771)
0808 11:44:39.951956 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:39.951964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:39.952675 (+ 711us) server_negotiation.cc:407] Connection header received
0808 11:44:39.952853 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:39.952856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:39.952913 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:39.952989 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:39.954421 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.955468 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.958228 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.958789 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.958921 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:39.961622 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:39.961631 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:39.961639 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:39.961686 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:39.965379 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:39.966371 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:39.966378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:39.966381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:39.966594 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:39.967073 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:39.967080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:39.967084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:39.967409 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:39.967560 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:39.967953 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 11:44:39.968175 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:44:39.968688 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:39.951736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46771 (local address 127.13.158.66:40215)
0808 11:44:39.952536 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:39.952562 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:39.952737 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:39.953057 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:39.953064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:39.953775 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:39.954229 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:39.954247 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.955650 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.955660 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:39.958037 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:39.958062 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:39.959130 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:39.959139 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:39.959376 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:39.960115 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:39.960145 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:39.961380 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:39.965507 (+ 4127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:39.965513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:39.965517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:39.966220 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:39.966782 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:39.966787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:39.966794 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:39.966937 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:39.967539 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:39.967544 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:39.967720 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 11:44:39.967860 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 11:44:39.968187 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":425,"threads_started":1}
I20260808 11:44:39.976933 25118 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:41331
I20260808 11:44:39.976961 25293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:41331 every 8 connection(s)
I20260808 11:44:39.978102 25118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:39.980065 25294 heartbeater.cc:347] Connected to a master server at 127.13.158.126:46771
I20260808 11:44:39.980258 25294 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:39.980859 25294 heartbeater.cc:511] Master 127.13.158.126:46771 requested a full tablet report, sending...
I20260808 11:44:39.982149 24823 ts_manager.cc:194] Registered new tserver with Master: fd4cb1cfaa864c988999beb92695e272 (127.13.158.66:41331)
I20260808 11:44:39.983055 24823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:53495
I20260808 11:44:39.985638 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 25118
I20260808 11:44:39.985816 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:39.986254 13945 external_mini_cluster.cc:1428] Setting key 25bb0c3e4d2ee5717ca043a3058092ac
WARNING: no policy specified for oryx/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab.
Entry for principal oryx/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab.
I20260808 11:44:40.041234 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:46771
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab
--principal=oryx/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:40.364292 25301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:40.364573 25301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:40.364722 25301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:40.371680 25301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:40.371774 25301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:40.371865 25301 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:44:40.371944 25301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:40.381932 25301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab
--principal=oryx/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.158.126:46771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:40.383047 25301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:40.384268 25301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:40.396951 25311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:40.397099 25309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:40.397404 25313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:40.397770 25301 server_base.cc:1034] running on GCE node
I20260808 11:44:40.398348 25301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:40.399240 25301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:40.400445 25301 hybrid_clock.cc:648] HybridClock initialized: now 1786189480400386 us; error 40 us; skew 500 ppm
I20260808 11:44:40.400700 25301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:40 dist-test-slave-39l8 krb5kdc[24757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189480, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:40.403916 25301 init.cc:377] Logged in from keytab as oryx/127.13.158.67@KRBTEST.COM (short username oryx)
I20260808 11:44:40.405251 25301 webserver.cc:460] Webserver started at http://127.13.158.67:42989/ using document root <none> and password file <none>
I20260808 11:44:40.405921 25301 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:40.406068 25301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:40.406402 25301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:40.409269 25301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "cbbbb6b12fc44c4285a5d7219e688beb"
format_stamp: "Formatted at 2026-08-08 11:44:40 on dist-test-slave-39l8"
server_key: "82ae625dc4519bced77a0b205969e2ac"
server_key_iv: "6bc4e02d5cb66585cf5001ad789894fd"
server_key_version: "encryptionkey@0"
I20260808 11:44:40.410048 25301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "cbbbb6b12fc44c4285a5d7219e688beb"
format_stamp: "Formatted at 2026-08-08 11:44:40 on dist-test-slave-39l8"
server_key: "82ae625dc4519bced77a0b205969e2ac"
server_key_iv: "6bc4e02d5cb66585cf5001ad789894fd"
server_key_version: "encryptionkey@0"
I20260808 11:44:40.415068 25301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:44:40.418310 25321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:40.419479 25301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:44:40.419651 25301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "cbbbb6b12fc44c4285a5d7219e688beb"
format_stamp: "Formatted at 2026-08-08 11:44:40 on dist-test-slave-39l8"
server_key: "82ae625dc4519bced77a0b205969e2ac"
server_key_iv: "6bc4e02d5cb66585cf5001ad789894fd"
server_key_version: "encryptionkey@0"
I20260808 11:44:40.419862 25301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:40.432843 25301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:40.436896 25301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:40.437170 25301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:40.439771 25301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:40.439879 25301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:40.440057 25301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:40.440146 25301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:44:40 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189480, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:40.465183 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.440243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:41415 (local address 127.13.158.126:46771)
0808 11:44:40.440502 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.440511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.441912 (+ 1401us) server_negotiation.cc:407] Connection header received
0808 11:44:40.444148 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.444155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.444264 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.444376 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:40.446473 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.448041 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.451449 (+ 3408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.451870 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.452035 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.457337 (+ 5302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:40.457345 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:40.457350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:40.457394 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:40.461098 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:40.462111 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:40.462116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:40.462120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:40.462276 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:40.462961 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:40.462965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:40.462968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:40.463377 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:40.463567 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.464579 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.464824 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:44:40.466284 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.440853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46771 (local address 127.13.158.67:41415)
0808 11:44:40.441705 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.441760 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.443870 (+ 2110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.444633 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.444649 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.445435 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:40.446270 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.446290 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.448293 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.448298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.451267 (+ 2969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.451289 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.452171 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.452179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.452484 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.453207 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:40.453246 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:40.456926 (+ 3680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:40.461260 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:40.461269 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:40.461286 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:40.461924 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:40.462411 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:40.462416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:40.462420 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:40.462743 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:40.463560 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:40.463568 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:40.464069 (+ 501us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.464422 (+ 353us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.465005 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":394,"threads_started":1}
I20260808 11:44:40.467334 25114 heartbeater.cc:503] Master 127.13.158.126:46771 was elected leader, sending a full tablet report...
I20260808 11:44:40.488720 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.472984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:44843 (local address 127.13.158.126:46771)
0808 11:44:40.473156 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.473162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.473337 (+ 175us) server_negotiation.cc:407] Connection header received
0808 11:44:40.473617 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.473622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.473714 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.473790 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:40.474776 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.475955 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.478664 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.479178 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.479276 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.481449 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:40.481458 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:40.481464 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:40.481510 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:40.485391 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:40.486416 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:40.486422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:40.486428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:40.486601 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:40.487110 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:40.487113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:40.487115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:40.487361 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:40.487464 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.488043 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.488321 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:44:40.488977 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.472658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46771 (local address 127.13.158.67:44843)
0808 11:44:40.473242 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.473263 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.473443 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.473937 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.473940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.474338 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:40.474607 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.474615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.476112 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.476119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.478467 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.478486 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.479380 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.479388 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.479612 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.480372 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:40.480400 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:40.481237 (+ 837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:40.485547 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:40.485554 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:40.485560 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:40.486281 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:40.486775 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:40.486783 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:40.486789 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:40.486982 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:40.487499 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:40.487504 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:40.487742 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.487927 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.488405 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":511,"threads_started":1}
I20260808 11:44:40.489362 25301 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:45733
I20260808 11:44:40.489383 25501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:45733 every 8 connection(s)
I20260808 11:44:40.490697 25301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:40.492848 25502 heartbeater.cc:347] Connected to a master server at 127.13.158.126:46771
I20260808 11:44:40.493062 25502 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:40.493572 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 25301
I20260808 11:44:40.493635 25502 heartbeater.cc:511] Master 127.13.158.126:46771 requested a full tablet report, sending...
I20260808 11:44:40.493762 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:40.494313 13945 external_mini_cluster.cc:1428] Setting key a8844877ee7bb1e4fd50210a7343c886
I20260808 11:44:40.495107 24823 ts_manager.cc:194] Registered new tserver with Master: cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:40.495913 24823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:41415
I20260808 11:44:40.497360 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:44:40 dist-test-slave-39l8 krb5kdc[24757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189480, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:44:40 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:40 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:44:40.550827 24967 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:40.540678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.13.158.126:46771)
0808 11:44:40.540925 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.540931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.541093 (+ 162us) server_negotiation.cc:407] Connection header received
0808 11:44:40.541258 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.541262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.541329 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.541416 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:40.542697 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.543495 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.544687 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.545133 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.545225 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.550490 (+ 5265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46702: BlockingRecv error: recv got EOF from 127.0.0.1:46702 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 08 11:44:40 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189480, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:40.567215 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.553987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.13.158.126:46771)
0808 11:44:40.554211 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.554219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.554505 (+ 286us) server_negotiation.cc:407] Connection header received
0808 11:44:40.554743 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.554749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.554832 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.554939 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:40.556101 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.556861 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.558049 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.558452 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.558587 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.562013 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:40.562018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:40.562023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:40.562058 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:40.564653 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:40.565304 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:40.565311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:40.565314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:40.565443 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:40.565819 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:40.565824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:40.565827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:40.566119 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:40.566387 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.566674 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.566941 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:44:40.571133 24823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:44:40.577935 24823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:40.600620 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.590295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57540 (local address 127.13.158.67:45733)
0808 11:44:40.591912 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.591923 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.591962 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:44:40.592088 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.592095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.592347 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.593575 (+ 1228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:40.593915 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.595580 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.598057 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.599432 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.599664 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.599918 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.600129 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.600287 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1403,"thread_start_us":1426,"threads_started":1}
I20260808 11:44:40.600711 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.589930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:45733 (local address 127.0.0.1:57540)
0808 11:44:40.590948 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.590995 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.591225 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.592602 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.592610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.592641 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:40.593160 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.593175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.595735 (+ 2560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.595743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.597900 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.597914 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.599674 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.599680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.599879 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.599932 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.600089 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.600260 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":1,"thread_start_us":894,"threads_started":1}
I20260808 11:44:40.601536 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.590842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.13.158.66:41331)
0808 11:44:40.591793 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.591802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.592512 (+ 710us) server_negotiation.cc:407] Connection header received
0808 11:44:40.592634 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.592639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.592784 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.593022 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:40.593942 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.595532 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.598361 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.599640 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.599761 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.599918 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.601075 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.601263 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":691,"threads_started":1}
I20260808 11:44:40.602012 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.590211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41331 (local address 127.0.0.1:48062)
0808 11:44:40.592354 (+ 2143us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.592372 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.592492 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.593262 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.593266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.593296 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:40.593750 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.593763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.595720 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.595727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.598198 (+ 2471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.598211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.600077 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.600083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.600242 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.600272 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.600359 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.600487 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1989,"mutex_wait_us":81,"spinlock_wait_cycles":3328,"thread_start_us":1856,"threads_started":1}
I20260808 11:44:40.602304 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.592647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:44101 (local address 127.0.0.1:56734)
0808 11:44:40.593262 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.593278 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.593380 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.595376 (+ 1996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.595393 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.595418 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:40.595910 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.595929 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.597931 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.597936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.599360 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.599372 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.601298 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.601304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.601459 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.601489 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.601572 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.601680 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":652,"threads_started":1}
I20260808 11:44:40.602689 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.593160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.13.158.65:44101)
0808 11:44:40.594937 (+ 1777us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.594947 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.594973 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:44:40.595048 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.595052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.595185 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.595397 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:40.596197 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.597767 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.599517 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.601007 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.602021 (+ 1014us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.602228 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.602301 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.602436 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1645,"thread_start_us":1536,"threads_started":1}
I20260808 11:44:40.604763 25401 tablet_service.cc:1467] Processing CreateTablet for tablet cd566f915fe04fa7beecc28b1a9ce6ab (DEFAULT_TABLE table=test-table [id=7766e59014464cf3a556b8bcd972db8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:40.606277 25401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd566f915fe04fa7beecc28b1a9ce6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:40.605793 25202 tablet_service.cc:1467] Processing CreateTablet for tablet cd566f915fe04fa7beecc28b1a9ce6ab (DEFAULT_TABLE table=test-table [id=7766e59014464cf3a556b8bcd972db8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:40.605827 25017 tablet_service.cc:1467] Processing CreateTablet for tablet cd566f915fe04fa7beecc28b1a9ce6ab (DEFAULT_TABLE table=test-table [id=7766e59014464cf3a556b8bcd972db8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:40.607075 25202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd566f915fe04fa7beecc28b1a9ce6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:40.607163 25017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd566f915fe04fa7beecc28b1a9ce6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:40.617274 25534 tablet_bootstrap.cc:492] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Bootstrap starting.
I20260808 11:44:40.620560 25535 tablet_bootstrap.cc:492] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Bootstrap starting.
I20260808 11:44:40.621940 25534 tablet_bootstrap.cc:654] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:40.622859 25536 tablet_bootstrap.cc:492] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Bootstrap starting.
I20260808 11:44:40.623262 25534 log.cc:824] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:40.623889 25535 tablet_bootstrap.cc:654] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:40.625057 25535 log.cc:824] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:40.626423 25534 tablet_bootstrap.cc:492] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: No bootstrap required, opened a new log
I20260808 11:44:40.626578 25536 tablet_bootstrap.cc:654] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:40.626750 25534 ts_tablet_manager.cc:1397] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 11:44:40.627828 25536 log.cc:824] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:40.628320 25535 tablet_bootstrap.cc:492] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: No bootstrap required, opened a new log
I20260808 11:44:40.628656 25535 ts_tablet_manager.cc:1397] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 11:44:40.634089 25534 raft_consensus.cc:348] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.634424 25534 raft_consensus.cc:374] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:40.634531 25534 raft_consensus.cc:729] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85290b6a4c444b2f83e0e91d67a7ac2a, State: Initialized, Role: FOLLOWER
I20260808 11:44:40.635200 25534 consensus_queue.cc:260] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.636751 25534 ts_tablet_manager.cc:1428] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent starting tablet: real 0.010s user 0.004s sys 0.005s
I20260808 11:44:40.637117 25536 tablet_bootstrap.cc:492] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: No bootstrap required, opened a new log
I20260808 11:44:40.637409 25536 ts_tablet_manager.cc:1397] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260808 11:44:40.640580 25535 raft_consensus.cc:348] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.640972 25535 raft_consensus.cc:374] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:40.641072 25535 raft_consensus.cc:729] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Initialized, Role: FOLLOWER
I20260808 11:44:40.641630 25535 consensus_queue.cc:260] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.642689 25294 heartbeater.cc:503] Master 127.13.158.126:46771 was elected leader, sending a full tablet report...
I20260808 11:44:40.643311 25535 ts_tablet_manager.cc:1428] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260808 11:44:40.645896 25536 raft_consensus.cc:348] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.646247 25536 raft_consensus.cc:374] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:40.646349 25536 raft_consensus.cc:729] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbbb6b12fc44c4285a5d7219e688beb, State: Initialized, Role: FOLLOWER
I20260808 11:44:40.646915 25536 consensus_queue.cc:260] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.647830 25502 heartbeater.cc:503] Master 127.13.158.126:46771 was elected leader, sending a full tablet report...
I20260808 11:44:40.648303 25536 ts_tablet_manager.cc:1428] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
W20260808 11:44:40.710240 25115 tablet.cc:2369] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:44:40.730062 25295 tablet.cc:2369] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:44:40.743005 25505 tablet.cc:2369] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:44:40.949903 25544 raft_consensus.cc:482] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:40.950301 25544 raft_consensus.cc:504] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.952093 25544 leader_election.cc:288] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733), 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101)
I20260808 11:44:40.959224 25543 raft_consensus.cc:482] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:40.959656 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.952557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:48849 (local address 127.13.158.67:45733)
0808 11:44:40.952776 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.952781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.953197 (+ 416us) server_negotiation.cc:407] Connection header received
0808 11:44:40.953430 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.953434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.953495 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.953597 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:40.954468 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.955445 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.957653 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.958725 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.958882 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.958952 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.959283 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.959403 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:44:40.959745 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.952411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:45733 (local address 127.13.158.66:48849)
0808 11:44:40.953073 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.953090 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.953263 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.953863 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.953868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.953901 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:40.954294 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.954303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.955571 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.955574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.957503 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.957517 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.958936 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.958943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.959111 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.959136 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.959246 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.959367 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":1,"thread_start_us":366,"threads_started":1}
I20260808 11:44:40.959558 25543 raft_consensus.cc:504] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.960855 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.953713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:34967 (local address 127.13.158.65:44101)
0808 11:44:40.953892 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.953897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.954541 (+ 644us) server_negotiation.cc:407] Connection header received
0808 11:44:40.954651 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.954655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.954777 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.954872 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:40.955371 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.956148 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.958314 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.959597 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.959778 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.959856 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.960488 (+ 632us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.960623 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:44:40.960875 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.953275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:44101 (local address 127.13.158.66:34967)
0808 11:44:40.954415 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.954438 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.954554 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.954903 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.954905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.954923 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:40.955219 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.955225 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.956300 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.956305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.958169 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.958180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.960153 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.960158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.960317 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.960351 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.960432 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.960539 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":773,"threads_started":1}
I20260808 11:44:40.960870 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd566f915fe04fa7beecc28b1a9ce6ab" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" is_pre_election: true
I20260808 11:44:40.961390 25428 raft_consensus.cc:2454] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 0.
I20260808 11:44:40.961710 25543 leader_election.cc:288] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733), fd4cb1cfaa864c988999beb92695e272 (127.13.158.66:41331)
I20260808 11:44:40.961920 25049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd566f915fe04fa7beecc28b1a9ce6ab" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" is_pre_election: true
I20260808 11:44:40.962155 25136 leader_election.cc:302] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbbbb6b12fc44c4285a5d7219e688beb, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:40.962433 25049 raft_consensus.cc:2454] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 0.
I20260808 11:44:40.962570 25544 raft_consensus.cc:2781] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:40.962762 25544 raft_consensus.cc:482] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:40.962889 25544 raft_consensus.cc:3037] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:40.966298 25544 raft_consensus.cc:504] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.966996 25544 leader_election.cc:288] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 election: Requested vote from peers cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733), 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101)
I20260808 11:44:40.967547 25049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd566f915fe04fa7beecc28b1a9ce6ab" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a"
I20260808 11:44:40.967798 25049 raft_consensus.cc:3037] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:40.969965 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd566f915fe04fa7beecc28b1a9ce6ab" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb"
I20260808 11:44:40.970229 25428 raft_consensus.cc:3037] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:40.971098 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.962157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:60181 (local address 127.13.158.66:41331)
0808 11:44:40.962344 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.962349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.964054 (+ 1705us) server_negotiation.cc:407] Connection header received
0808 11:44:40.964323 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.964327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.964403 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.964491 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:40.965270 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.966072 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.968677 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.969917 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.970079 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.970149 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.970729 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.970865 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:44:40.971275 25049 raft_consensus.cc:2454] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 1.
I20260808 11:44:40.971683 25136 leader_election.cc:302] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85290b6a4c444b2f83e0e91d67a7ac2a, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:40.971805 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.962167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41331 (local address 127.13.158.65:60181)
0808 11:44:40.963933 (+ 1766us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.963955 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.964161 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.964694 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.964698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.964724 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:40.965061 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.965073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.966245 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.966249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.968513 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.968527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.970321 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.970327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.970499 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.970540 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.971256 (+ 716us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.971401 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1599,"mutex_wait_us":16,"thread_start_us":1706,"threads_started":1}
I20260808 11:44:40.972101 25544 raft_consensus.cc:2781] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:40.972771 25236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd566f915fe04fa7beecc28b1a9ce6ab" candidate_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd4cb1cfaa864c988999beb92695e272" is_pre_election: true
I20260808 11:44:40.973477 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.964485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:51895 (local address 127.13.158.67:45733)
0808 11:44:40.964675 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:40.964681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:40.964699 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:44:40.964755 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:40.964759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:40.964824 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:40.964906 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:44:40.965581 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.966639 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.969638 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.970977 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.971141 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.971546 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:40.972914 (+ 1368us) server_negotiation.cc:299] Negotiation successful
0808 11:44:40.973105 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 11:44:40.973557 25428 raft_consensus.cc:2454] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 1.
I20260808 11:44:40.974128 25544 raft_consensus.cc:686] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 1 LEADER]: Becoming Leader. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Running, Role: LEADER
I20260808 11:44:40.974427 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:40.962894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:45733 (local address 127.13.158.65:51895)
0808 11:44:40.964222 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:40.964240 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:40.964346 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:40.965029 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:40.965034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:40.965062 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:44:40.965427 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.965438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.967065 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.967130 (+ 65us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.969452 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:40.969467 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:40.971764 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:40.971771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:40.972654 (+ 883us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:40.972712 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 11:44:40.973909 (+ 1197us) client_negotiation.cc:240] Negotiation successful
0808 11:44:40.974052 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1202,"mutex_wait_us":29,"thread_start_us":637,"threads_started":1}
I20260808 11:44:40.974695 25544 consensus_queue.cc:237] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } }
I20260808 11:44:40.974987 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd566f915fe04fa7beecc28b1a9ce6ab" candidate_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" is_pre_election: true
I20260808 11:44:40.975291 25428 raft_consensus.cc:2379] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85290b6a4c444b2f83e0e91d67a7ac2a in current term 1: Already voted for candidate fd4cb1cfaa864c988999beb92695e272 in this term.
I20260808 11:44:40.979372 24961 leader_election.cc:302] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85290b6a4c444b2f83e0e91d67a7ac2a; no voters: cbbbb6b12fc44c4285a5d7219e688beb, fd4cb1cfaa864c988999beb92695e272
I20260808 11:44:40.979818 25543 raft_consensus.cc:2726] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 11:44:40.980687 24823 catalog_manager.cc:5495] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd4cb1cfaa864c988999beb92695e272 (127.13.158.66). New cstate: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } health_report { overall_health: UNKNOWN } } }
I20260808 11:44:41.053351 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:41.048317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.13.158.66:41331)
0808 11:44:41.048464 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:41.048468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:41.048603 (+ 135us) server_negotiation.cc:407] Connection header received
0808 11:44:41.048792 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:41.048796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:41.048872 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:41.048995 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:41.049349 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.050072 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.051380 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.051808 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.051935 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.052331 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:41.052836 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:41.053008 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:41.053047 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 11:44:41.053112 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 11:44:41.061353 25428 raft_consensus.cc:1261] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Refusing update from remote peer fd4cb1cfaa864c988999beb92695e272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:41.061347 25049 raft_consensus.cc:1261] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Refusing update from remote peer fd4cb1cfaa864c988999beb92695e272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:41.062150 25552 consensus_queue.cc:1035] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:41.062485 25544 consensus_queue.cc:1035] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:41.073065 25556 mvcc.cc:205] Tried to move back new op lower bound from 7316232114418499584 to 7316232114077290496. Current Snapshot: MvccSnapshot[applied={T|T < 7316232114418499584}]
I20260808 11:44:41.074581 25558 mvcc.cc:205] Tried to move back new op lower bound from 7316232114418499584 to 7316232114077290496. Current Snapshot: MvccSnapshot[applied={T|T < 7316232114418499584}]
I20260808 11:44:41.087765 24822 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:41.088076 24822 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:41.091965 24822 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: Sending DeleteTablet for 3 replicas of tablet cd566f915fe04fa7beecc28b1a9ce6ab
I20260808 11:44:41.093206 25202 tablet_service.cc:1514] Processing DeleteTablet for tablet cd566f915fe04fa7beecc28b1a9ce6ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:41 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:48062
I20260808 11:44:41.093405 25401 tablet_service.cc:1514] Processing DeleteTablet for tablet cd566f915fe04fa7beecc28b1a9ce6ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:41 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:57540
I20260808 11:44:41.093457 25017 tablet_service.cc:1514] Processing DeleteTablet for tablet cd566f915fe04fa7beecc28b1a9ce6ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:41 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:56734
I20260808 11:44:41.098031 24822 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:44:41.099949 24822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:41.104921 25569 tablet_replica.cc:330] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: stopping tablet replica
I20260808 11:44:41.105597 25569 raft_consensus.cc:2229] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:41.106284 25569 raft_consensus.cc:2258] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:41.110314 25569 ts_tablet_manager.cc:1905] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:41.111071 25201 tablet_service.cc:1467] Processing CreateTablet for tablet e2374f1faec0495d8965626a4af6843f (DEFAULT_TABLE table=test-table [id=822e31d10a7343139216d1988ad7f764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:41.111747 25201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2374f1faec0495d8965626a4af6843f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.112922 25017 tablet_service.cc:1467] Processing CreateTablet for tablet e2374f1faec0495d8965626a4af6843f (DEFAULT_TABLE table=test-table [id=822e31d10a7343139216d1988ad7f764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:41.113381 25017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2374f1faec0495d8965626a4af6843f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.113950 25568 tablet_replica.cc:330] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: stopping tablet replica
I20260808 11:44:41.114418 25568 raft_consensus.cc:2229] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:41.114588 25567 tablet_replica.cc:330] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: stopping tablet replica
I20260808 11:44:41.114964 25568 raft_consensus.cc:2258] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:41.116539 25567 raft_consensus.cc:2229] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:44:41.119910 25534 tablet_bootstrap.cc:492] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: Bootstrap starting.
I20260808 11:44:41.121639 25534 tablet_bootstrap.cc:654] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:41.122246 25401 tablet_service.cc:1467] Processing CreateTablet for tablet e2374f1faec0495d8965626a4af6843f (DEFAULT_TABLE table=test-table [id=822e31d10a7343139216d1988ad7f764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:41.121851 25535 tablet_bootstrap.cc:492] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: Bootstrap starting.
I20260808 11:44:41.122784 25401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2374f1faec0495d8965626a4af6843f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.124414 25535 tablet_bootstrap.cc:654] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:41.119963 25567 raft_consensus.cc:2258] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:41.121097 25568 ts_tablet_manager.cc:1905] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:41.130259 25536 tablet_bootstrap.cc:492] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: Bootstrap starting.
I20260808 11:44:41.132627 25536 tablet_bootstrap.cc:654] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:41.133100 25534 tablet_bootstrap.cc:492] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: No bootstrap required, opened a new log
I20260808 11:44:41.133279 25534 ts_tablet_manager.cc:1397] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.002s
I20260808 11:44:41.134315 25534 raft_consensus.cc:348] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.134625 25534 raft_consensus.cc:374] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:41.134755 25534 raft_consensus.cc:729] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85290b6a4c444b2f83e0e91d67a7ac2a, State: Initialized, Role: FOLLOWER
I20260808 11:44:41.135146 25534 consensus_queue.cc:260] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.135877 25534 ts_tablet_manager.cc:1428] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 11:44:41.137732 25568 ts_tablet_manager.cc:1918] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:41.137889 25568 log.cc:1196] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/cd566f915fe04fa7beecc28b1a9ce6ab
I20260808 11:44:41.138413 25568 ts_tablet_manager.cc:1939] T cd566f915fe04fa7beecc28b1a9ce6ab P cbbbb6b12fc44c4285a5d7219e688beb: Deleting consensus metadata
I20260808 11:44:41.138805 25535 tablet_bootstrap.cc:492] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: No bootstrap required, opened a new log
I20260808 11:44:41.138968 25535 ts_tablet_manager.cc:1397] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.005s
I20260808 11:44:41.139961 25535 raft_consensus.cc:348] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.140210 25535 raft_consensus.cc:374] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:41.140311 25535 raft_consensus.cc:729] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Initialized, Role: FOLLOWER
I20260808 11:44:41.140267 24801 catalog_manager.cc:4841] TS cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733): tablet cd566f915fe04fa7beecc28b1a9ce6ab (table test-table [id=7766e59014464cf3a556b8bcd972db8c]) successfully deleted
I20260808 11:44:41.140604 25535 consensus_queue.cc:260] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.141235 25535 ts_tablet_manager.cc:1428] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 11:44:41.141503 25567 ts_tablet_manager.cc:1905] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:41.143187 25569 ts_tablet_manager.cc:1918] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:41.143363 25569 log.cc:1196] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/cd566f915fe04fa7beecc28b1a9ce6ab
I20260808 11:44:41.143894 25569 ts_tablet_manager.cc:1939] T cd566f915fe04fa7beecc28b1a9ce6ab P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting consensus metadata
I20260808 11:44:41.145537 24801 catalog_manager.cc:4841] TS 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101): tablet cd566f915fe04fa7beecc28b1a9ce6ab (table test-table [id=7766e59014464cf3a556b8bcd972db8c]) successfully deleted
I20260808 11:44:41.145833 25536 tablet_bootstrap.cc:492] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: No bootstrap required, opened a new log
I20260808 11:44:41.145983 25536 ts_tablet_manager.cc:1397] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.004s
I20260808 11:44:41.146582 25536 raft_consensus.cc:348] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.146816 25536 raft_consensus.cc:374] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:41.146914 25536 raft_consensus.cc:729] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbbb6b12fc44c4285a5d7219e688beb, State: Initialized, Role: FOLLOWER
I20260808 11:44:41.147156 25536 consensus_queue.cc:260] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.147708 25536 ts_tablet_manager.cc:1428] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:44:41.149309 25567 ts_tablet_manager.cc:1918] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:41.149712 25567 log.cc:1196] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/cd566f915fe04fa7beecc28b1a9ce6ab
I20260808 11:44:41.150296 25567 ts_tablet_manager.cc:1939] T cd566f915fe04fa7beecc28b1a9ce6ab P fd4cb1cfaa864c988999beb92695e272: Deleting consensus metadata
I20260808 11:44:41.151873 24804 catalog_manager.cc:4841] TS fd4cb1cfaa864c988999beb92695e272 (127.13.158.66:41331): tablet cd566f915fe04fa7beecc28b1a9ce6ab (table test-table [id=7766e59014464cf3a556b8bcd972db8c]) successfully deleted
I20260808 11:44:41.436654 25544 raft_consensus.cc:482] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:41.436914 25544 raft_consensus.cc:504] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.437606 25544 leader_election.cc:288] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101), cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:41.438344 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2374f1faec0495d8965626a4af6843f" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" is_pre_election: true
I20260808 11:44:41.438356 25049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2374f1faec0495d8965626a4af6843f" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" is_pre_election: true
I20260808 11:44:41.438584 25428 raft_consensus.cc:2454] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 0.
I20260808 11:44:41.438657 25049 raft_consensus.cc:2454] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 0.
I20260808 11:44:41.439108 25136 leader_election.cc:302] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbbbb6b12fc44c4285a5d7219e688beb, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:41.439391 25544 raft_consensus.cc:2781] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:41.439524 25544 raft_consensus.cc:482] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:41.439683 25544 raft_consensus.cc:3037] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:41.441828 25544 raft_consensus.cc:504] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.442368 25544 leader_election.cc:288] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 election: Requested vote from peers 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101), cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:41.442999 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2374f1faec0495d8965626a4af6843f" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb"
I20260808 11:44:41.442997 25049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2374f1faec0495d8965626a4af6843f" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a"
I20260808 11:44:41.443217 25428 raft_consensus.cc:3037] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:41.443213 25049 raft_consensus.cc:3037] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:41.445150 25428 raft_consensus.cc:2454] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 1.
I20260808 11:44:41.445536 25136 leader_election.cc:302] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbbbb6b12fc44c4285a5d7219e688beb, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:41.445828 25544 raft_consensus.cc:2781] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:41.445997 25544 raft_consensus.cc:686] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 1 LEADER]: Becoming Leader. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Running, Role: LEADER
I20260808 11:44:41.446332 25049 raft_consensus.cc:2454] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 1.
I20260808 11:44:41.446341 25544 consensus_queue.cc:237] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.449354 24822 catalog_manager.cc:5495] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd4cb1cfaa864c988999beb92695e272 (127.13.158.66). New cstate: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } health_report { overall_health: UNKNOWN } } }
Aug 08 11:44:41 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189478, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.126@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:41.530915 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:41.512820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48804 (local address 127.13.158.126:46771)
0808 11:44:41.513616 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:41.513620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:41.513644 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:44:41.513794 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:41.513798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:41.513857 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:41.513957 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:41.515076 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.515857 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.517003 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.517387 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.517474 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.525307 (+ 7833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:41.525311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:41.525314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:41.525340 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:41.528142 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:41.528801 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:41.528808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:41.528814 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:41.528931 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:41.529303 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:41.529309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:41.529312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:41.529630 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:41.529768 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:41.530172 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 11:44:41.530450 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":690,"thread_start_us":540,"threads_started":1}
I20260808 11:44:41.530995 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:41.512860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:46771 (local address 127.0.0.1:48804)
0808 11:44:41.513549 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:41.513565 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:41.513684 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:41.514006 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:41.514009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:41.514666 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:41.514929 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:41.514938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.515968 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.515971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:41.516895 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:41.516906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.517519 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.517523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:41.517624 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.522774 (+ 5150us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:41.522793 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:41.525113 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:41.528273 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:41.528285 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:41.528292 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:41.528685 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:41.529039 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:41.529044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:41.529047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:41.529188 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:41.529749 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:41.529755 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:41.529924 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 11:44:41.530096 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 11:44:41.530571 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":195,"thread_start_us":369,"threads_started":1}
I20260808 11:44:41.535820 24822 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:48804:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 11:44:41.536890 24822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:41.543305 25017 tablet_service.cc:1467] Processing CreateTablet for tablet 33829fdb4a8249579078aa4693d50aa5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4822f746ca1140a4859543d2eb478af6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:44:41.543545 25201 tablet_service.cc:1467] Processing CreateTablet for tablet 33829fdb4a8249579078aa4693d50aa5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4822f746ca1140a4859543d2eb478af6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:44:41.543924 25017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33829fdb4a8249579078aa4693d50aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.543855 25401 tablet_service.cc:1467] Processing CreateTablet for tablet 33829fdb4a8249579078aa4693d50aa5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4822f746ca1140a4859543d2eb478af6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:44:41.544106 25201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33829fdb4a8249579078aa4693d50aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.544371 25401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33829fdb4a8249579078aa4693d50aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.549595 25535 tablet_bootstrap.cc:492] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: Bootstrap starting.
I20260808 11:44:41.550745 25534 tablet_bootstrap.cc:492] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: Bootstrap starting.
I20260808 11:44:41.550976 25536 tablet_bootstrap.cc:492] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: Bootstrap starting.
I20260808 11:44:41.552373 25534 tablet_bootstrap.cc:654] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:41.552531 25536 tablet_bootstrap.cc:654] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:41.553339 25535 tablet_bootstrap.cc:654] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:41.555202 25536 tablet_bootstrap.cc:492] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: No bootstrap required, opened a new log
I20260808 11:44:41.555365 25536 ts_tablet_manager.cc:1397] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260808 11:44:41.555550 25535 tablet_bootstrap.cc:492] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: No bootstrap required, opened a new log
I20260808 11:44:41.555716 25535 ts_tablet_manager.cc:1397] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260808 11:44:41.556041 25536 raft_consensus.cc:348] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.556219 25536 raft_consensus.cc:374] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:41.556316 25536 raft_consensus.cc:729] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbbb6b12fc44c4285a5d7219e688beb, State: Initialized, Role: FOLLOWER
I20260808 11:44:41.556377 25535 raft_consensus.cc:348] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.556684 25535 raft_consensus.cc:374] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:41.556592 25536 consensus_queue.cc:260] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.556792 25535 raft_consensus.cc:729] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Initialized, Role: FOLLOWER
I20260808 11:44:41.557103 25536 ts_tablet_manager.cc:1428] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 11:44:41.557096 25535 consensus_queue.cc:260] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.557047 25545 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.557422 25545 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:44:41.557546 25544 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.557811 25544 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:44:41.558291 25535 ts_tablet_manager.cc:1428] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260808 11:44:41.558947 25534 tablet_bootstrap.cc:492] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: No bootstrap required, opened a new log
I20260808 11:44:41.559119 25534 ts_tablet_manager.cc:1397] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260808 11:44:41.559895 25534 raft_consensus.cc:348] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.560145 25534 raft_consensus.cc:374] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:41.560240 25534 raft_consensus.cc:729] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85290b6a4c444b2f83e0e91d67a7ac2a, State: Initialized, Role: FOLLOWER
I20260808 11:44:41.560473 25534 consensus_queue.cc:260] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.560998 25534 ts_tablet_manager.cc:1428] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:44:41.561296 25543 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.561528 25543 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:44:41.706338 25544 raft_consensus.cc:482] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:41.706558 25544 raft_consensus.cc:504] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.707162 25544 leader_election.cc:288] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101), cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:41.707732 25049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33829fdb4a8249579078aa4693d50aa5" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" is_pre_election: true
I20260808 11:44:41.707803 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33829fdb4a8249579078aa4693d50aa5" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" is_pre_election: true
I20260808 11:44:41.708010 25049 raft_consensus.cc:2454] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 0.
I20260808 11:44:41.708066 25428 raft_consensus.cc:2454] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 0.
I20260808 11:44:41.708487 25136 leader_election.cc:302] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbbbb6b12fc44c4285a5d7219e688beb, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:41.708765 25544 raft_consensus.cc:2781] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:41.708853 25544 raft_consensus.cc:482] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:41.708926 25544 raft_consensus.cc:3037] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:41.710850 25544 raft_consensus.cc:504] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.711270 25544 leader_election.cc:288] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 election: Requested vote from peers 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101), cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:41.711704 25049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33829fdb4a8249579078aa4693d50aa5" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a"
I20260808 11:44:41.711750 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33829fdb4a8249579078aa4693d50aa5" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb"
I20260808 11:44:41.711921 25049 raft_consensus.cc:3037] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:41.711948 25428 raft_consensus.cc:3037] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:41.714002 25049 raft_consensus.cc:2454] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 1.
I20260808 11:44:41.714232 25428 raft_consensus.cc:2454] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 1.
I20260808 11:44:41.714438 25136 leader_election.cc:302] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85290b6a4c444b2f83e0e91d67a7ac2a, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:41.714691 25544 raft_consensus.cc:2781] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:41.714949 25544 raft_consensus.cc:686] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [term 1 LEADER]: Becoming Leader. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Running, Role: LEADER
I20260808 11:44:41.715204 25544 consensus_queue.cc:237] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:41.716656 25552 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: TxnStatusTablet state changed. Reason: New leader fd4cb1cfaa864c988999beb92695e272. Latest consensus state: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.716920 25552 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:44:41.717446 25592 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 11:44:41.717661 25592 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 11:44:41.718174 24819 catalog_manager.cc:5495] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd4cb1cfaa864c988999beb92695e272 (127.13.158.66). New cstate: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } health_report { overall_health: UNKNOWN } } }
I20260808 11:44:41.747804 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:41.741903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46492 (local address 127.13.158.66:41331)
0808 11:44:41.742531 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:41.742536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:41.742551 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:44:41.742605 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:41.742609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:41.742672 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:41.742816 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:41.743273 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.744081 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.745929 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.746369 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.746461 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.747093 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:41.747190 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:41.747266 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:41.747482 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 11:44:41.747542 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":378,"threads_started":1}
I20260808 11:44:41.747947 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:41.741905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41331 (local address 127.0.0.1:46492)
0808 11:44:41.742165 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:41.742190 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:41.742335 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:41.742878 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:41.742883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:41.742897 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:44:41.743094 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:41.743106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.744222 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.744228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:41.745788 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:41.745805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.746524 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.746530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:41.746779 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.746862 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:44:41.747325 (+ 463us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:44:41.747347 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:44:41.747438 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 11:44:41.747586 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 11:44:41.752314 25428 raft_consensus.cc:1261] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Refusing update from remote peer fd4cb1cfaa864c988999beb92695e272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:41.752727 25049 raft_consensus.cc:1261] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Refusing update from remote peer fd4cb1cfaa864c988999beb92695e272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:41.752938 25552 consensus_queue.cc:1035] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:41.753319 25544 consensus_queue.cc:1035] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:41.759635 25545 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: TxnStatusTablet state changed. Reason: New leader fd4cb1cfaa864c988999beb92695e272. Latest consensus state: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.759917 25545 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:44:41.763546 25545 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.763849 25545 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P cbbbb6b12fc44c4285a5d7219e688beb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:44:41.766198 25544 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.766517 25544 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:44:41.767959 25543 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: TxnStatusTablet state changed. Reason: New leader fd4cb1cfaa864c988999beb92695e272. Latest consensus state: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.768232 25543 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:44:41.775559 25543 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.775794 25543 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P 85290b6a4c444b2f83e0e91d67a7ac2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:44:41.779232 25605 tablet_replica.cc:439] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } } }
I20260808 11:44:41.779633 25605 tablet_replica.cc:442] T 33829fdb4a8249579078aa4693d50aa5 P fd4cb1cfaa864c988999beb92695e272: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:44:41.794368 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:41.787766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:45931 (local address 127.13.158.66:41331)
0808 11:44:41.787959 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:41.787967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:41.788587 (+ 620us) server_negotiation.cc:407] Connection header received
0808 11:44:41.788735 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:41.788740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:41.788834 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:41.788927 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:41.789413 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.790567 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.792264 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.792909 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.793031 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.793618 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:41.793736 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:41.793836 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:41.793961 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 11:44:41.794039 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":58}
I20260808 11:44:41.794387 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:41.787784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41331 (local address 127.13.158.66:45931)
0808 11:44:41.788494 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:41.788509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:41.788623 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:41.788945 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:41.788948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:41.788956 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:44:41.789249 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:41.789257 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.790684 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.790688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:41.792112 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:41.792124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.793053 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.793058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:41.793350 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.793434 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:44:41.793845 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:44:41.793855 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 11:44:41.793917 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 11:44:41.794013 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":45,"thread_start_us":350,"threads_started":1}
I20260808 11:44:41.803498 25428 raft_consensus.cc:1261] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Refusing update from remote peer fd4cb1cfaa864c988999beb92695e272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:41.803493 25049 raft_consensus.cc:1261] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Refusing update from remote peer fd4cb1cfaa864c988999beb92695e272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:41.804080 25552 consensus_queue.cc:1035] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:41.804402 25605 consensus_queue.cc:1035] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:41.870877 24823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:41.871133 24823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:41.874030 24823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: Sending DeleteTablet for 3 replicas of tablet e2374f1faec0495d8965626a4af6843f
I20260808 11:44:41.874929 25401 tablet_service.cc:1514] Processing DeleteTablet for tablet e2374f1faec0495d8965626a4af6843f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:41 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:57540
I20260808 11:44:41.874938 25202 tablet_service.cc:1514] Processing DeleteTablet for tablet e2374f1faec0495d8965626a4af6843f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:41 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:48062
I20260808 11:44:41.875379 25017 tablet_service.cc:1514] Processing DeleteTablet for tablet e2374f1faec0495d8965626a4af6843f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:41 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:56734
I20260808 11:44:41.875615 25618 tablet_replica.cc:330] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: stopping tablet replica
I20260808 11:44:41.875821 25618 raft_consensus.cc:2229] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:41.876103 25621 tablet_replica.cc:330] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: stopping tablet replica
I20260808 11:44:41.876163 25618 raft_consensus.cc:2258] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:41.876379 25621 raft_consensus.cc:2229] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:41.877010 25621 raft_consensus.cc:2258] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:41.877511 25618 ts_tablet_manager.cc:1905] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:41.878176 25621 ts_tablet_manager.cc:1905] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:41.884665 25618 ts_tablet_manager.cc:1918] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:44:41.884879 25618 log.cc:1196] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/e2374f1faec0495d8965626a4af6843f
I20260808 11:44:41.885430 25618 ts_tablet_manager.cc:1939] T e2374f1faec0495d8965626a4af6843f P cbbbb6b12fc44c4285a5d7219e688beb: Deleting consensus metadata
I20260808 11:44:41.886328 25621 ts_tablet_manager.cc:1918] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:44:41.886533 25621 log.cc:1196] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/e2374f1faec0495d8965626a4af6843f
I20260808 11:44:41.886581 24801 catalog_manager.cc:4841] TS cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733): tablet e2374f1faec0495d8965626a4af6843f (table test-table [id=822e31d10a7343139216d1988ad7f764]) successfully deleted
I20260808 11:44:41.887084 25621 ts_tablet_manager.cc:1939] T e2374f1faec0495d8965626a4af6843f P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting consensus metadata
I20260808 11:44:41.888165 24801 catalog_manager.cc:4841] TS 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101): tablet e2374f1faec0495d8965626a4af6843f (table test-table [id=822e31d10a7343139216d1988ad7f764]) successfully deleted
I20260808 11:44:41.895326 25619 tablet_replica.cc:330] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: stopping tablet replica
I20260808 11:44:41.895548 25619 raft_consensus.cc:2229] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:44:41.896003 25619 raft_consensus.cc:2258] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:41.897023 25619 ts_tablet_manager.cc:1905] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:41.904803 25619 ts_tablet_manager.cc:1918] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:44:41.904946 25619 log.cc:1196] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/e2374f1faec0495d8965626a4af6843f
I20260808 11:44:41.905426 25619 ts_tablet_manager.cc:1939] T e2374f1faec0495d8965626a4af6843f P fd4cb1cfaa864c988999beb92695e272: Deleting consensus metadata
I20260808 11:44:41.906594 24804 catalog_manager.cc:4841] TS fd4cb1cfaa864c988999beb92695e272 (127.13.158.66:41331): tablet e2374f1faec0495d8965626a4af6843f (table test-table [id=822e31d10a7343139216d1988ad7f764]) successfully deleted
Aug 08 11:44:41 dist-test-slave-39l8 krb5kdc[24757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:44:41 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:41 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:44:41.944386 25583 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:41.933651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.13.158.126:46771)
0808 11:44:41.933872 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:41.933880 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:41.934240 (+ 360us) server_negotiation.cc:407] Connection header received
0808 11:44:41.934395 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:41.934403 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:41.934490 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:41.934611 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:41.935962 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.937193 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.938398 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.939070 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.939211 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.944124 (+ 4913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48806: BlockingRecv error: recv got EOF from 127.0.0.1:48806 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 11:44:41 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM
I20260808 11:44:41.962242 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:41.947134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48822 (local address 127.13.158.126:46771)
0808 11:44:41.947294 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:41.947299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:41.947625 (+ 326us) server_negotiation.cc:407] Connection header received
0808 11:44:41.947784 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:41.947790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:41.947866 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:41.947970 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:41.949245 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.950343 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.951582 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:41.952255 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:41.952378 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:41.956074 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:41.956081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:41.956086 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:41.956170 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:41.959708 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:41.960381 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:41.960386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:41.960390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:41.960547 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:41.960916 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:41.960921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:41.960924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:41.961246 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:41.961505 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:41.961688 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 11:44:41.961902 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:44:41.965831 24823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:44:41.966456 24823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:41.973625 25017 tablet_service.cc:1467] Processing CreateTablet for tablet cef7b2dee74249aca79b7261b2b8321e (DEFAULT_TABLE table=test-table [id=cd7d43c2254f442990b6f76ed601ccad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:41.974059 25017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef7b2dee74249aca79b7261b2b8321e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.979226 25202 tablet_service.cc:1467] Processing CreateTablet for tablet cef7b2dee74249aca79b7261b2b8321e (DEFAULT_TABLE table=test-table [id=cd7d43c2254f442990b6f76ed601ccad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:41.979671 25202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef7b2dee74249aca79b7261b2b8321e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.983330 25401 tablet_service.cc:1467] Processing CreateTablet for tablet cef7b2dee74249aca79b7261b2b8321e (DEFAULT_TABLE table=test-table [id=cd7d43c2254f442990b6f76ed601ccad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:41.983770 25401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef7b2dee74249aca79b7261b2b8321e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:41.987093 25534 tablet_bootstrap.cc:492] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: Bootstrap starting.
I20260808 11:44:41.988786 25534 tablet_bootstrap.cc:654] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:41.991742 25536 tablet_bootstrap.cc:492] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: Bootstrap starting.
I20260808 11:44:41.991767 25535 tablet_bootstrap.cc:492] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: Bootstrap starting.
I20260808 11:44:41.993477 25536 tablet_bootstrap.cc:654] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:41.993568 25535 tablet_bootstrap.cc:654] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:42.005087 25535 tablet_bootstrap.cc:492] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: No bootstrap required, opened a new log
I20260808 11:44:42.005402 25535 ts_tablet_manager.cc:1397] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260808 11:44:42.006606 25536 tablet_bootstrap.cc:492] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: No bootstrap required, opened a new log
I20260808 11:44:42.006392 25535 raft_consensus.cc:348] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.006732 25535 raft_consensus.cc:374] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:42.006870 25535 raft_consensus.cc:729] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Initialized, Role: FOLLOWER
I20260808 11:44:42.006953 25536 ts_tablet_manager.cc:1397] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.010s
I20260808 11:44:42.007318 25535 consensus_queue.cc:260] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.007827 25536 raft_consensus.cc:348] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.008062 25536 raft_consensus.cc:374] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:42.008147 25536 raft_consensus.cc:729] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbbb6b12fc44c4285a5d7219e688beb, State: Initialized, Role: FOLLOWER
I20260808 11:44:42.008867 25536 consensus_queue.cc:260] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.010675 25535 ts_tablet_manager.cc:1428] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260808 11:44:42.013175 25536 ts_tablet_manager.cc:1428] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260808 11:44:42.020375 25534 tablet_bootstrap.cc:492] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: No bootstrap required, opened a new log
I20260808 11:44:42.020608 25534 ts_tablet_manager.cc:1397] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.008s
I20260808 11:44:42.021260 25534 raft_consensus.cc:348] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.021481 25534 raft_consensus.cc:374] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:42.021579 25534 raft_consensus.cc:729] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85290b6a4c444b2f83e0e91d67a7ac2a, State: Initialized, Role: FOLLOWER
I20260808 11:44:42.021867 25534 consensus_queue.cc:260] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.022401 25534 ts_tablet_manager.cc:1428] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 11:44:42.152382 25544 raft_consensus.cc:482] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:42.152606 25544 raft_consensus.cc:504] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.153177 25544 leader_election.cc:288] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101), cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:42.153849 25047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef7b2dee74249aca79b7261b2b8321e" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" is_pre_election: true
I20260808 11:44:42.153870 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef7b2dee74249aca79b7261b2b8321e" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" is_pre_election: true
I20260808 11:44:42.154098 25047 raft_consensus.cc:2454] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 0.
I20260808 11:44:42.154147 25428 raft_consensus.cc:2454] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 0.
I20260808 11:44:42.154613 25136 leader_election.cc:302] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85290b6a4c444b2f83e0e91d67a7ac2a, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:42.154963 25544 raft_consensus.cc:2781] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:42.155079 25544 raft_consensus.cc:482] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:42.155215 25544 raft_consensus.cc:3037] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:42.157399 25544 raft_consensus.cc:504] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.157976 25544 leader_election.cc:288] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 election: Requested vote from peers 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101), cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:42.158478 25047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef7b2dee74249aca79b7261b2b8321e" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a"
I20260808 11:44:42.158514 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef7b2dee74249aca79b7261b2b8321e" candidate_uuid: "fd4cb1cfaa864c988999beb92695e272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb"
I20260808 11:44:42.158670 25047 raft_consensus.cc:3037] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:42.158756 25428 raft_consensus.cc:3037] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:42.160605 25428 raft_consensus.cc:2454] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 1.
I20260808 11:44:42.160660 25047 raft_consensus.cc:2454] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4cb1cfaa864c988999beb92695e272 in term 1.
I20260808 11:44:42.161056 25136 leader_election.cc:302] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85290b6a4c444b2f83e0e91d67a7ac2a, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:42.161327 25544 raft_consensus.cc:2781] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:42.161540 25544 raft_consensus.cc:686] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 1 LEADER]: Becoming Leader. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Running, Role: LEADER
I20260808 11:44:42.161834 25544 consensus_queue.cc:237] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.164589 24823 catalog_manager.cc:5495] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd4cb1cfaa864c988999beb92695e272 (127.13.158.66). New cstate: current_term: 1 leader_uuid: "fd4cb1cfaa864c988999beb92695e272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } health_report { overall_health: UNKNOWN } } }
I20260808 11:44:42.180526 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.175585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46502 (local address 127.13.158.66:41331)
0808 11:44:42.175800 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:42.175806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:42.175929 (+ 123us) server_negotiation.cc:407] Connection header received
0808 11:44:42.176152 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:42.176155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:42.176219 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:42.176317 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:42.176699 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.177428 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.178756 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.179188 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.179330 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.179788 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:42.179884 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:42.179986 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:42.180147 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 11:44:42.180225 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:44:42.183521 25428 raft_consensus.cc:1261] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Refusing update from remote peer fd4cb1cfaa864c988999beb92695e272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:42.183521 25047 raft_consensus.cc:1261] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Refusing update from remote peer fd4cb1cfaa864c988999beb92695e272: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:42.184109 25552 consensus_queue.cc:1035] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:42.184438 25544 consensus_queue.cc:1035] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:42.190892 25598 mvcc.cc:205] Tried to move back new op lower bound from 7316232119016771584 to 7316232118936997888. Current Snapshot: MvccSnapshot[applied={T|T < 7316232119016771584}]
I20260808 11:44:42.196117 24822 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:42.196358 24822 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:42.199038 24822 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: Sending DeleteTablet for 3 replicas of tablet cef7b2dee74249aca79b7261b2b8321e
I20260808 11:44:42.199791 25017 tablet_service.cc:1514] Processing DeleteTablet for tablet cef7b2dee74249aca79b7261b2b8321e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:42 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:56734
I20260808 11:44:42.199877 25202 tablet_service.cc:1514] Processing DeleteTablet for tablet cef7b2dee74249aca79b7261b2b8321e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:42 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:48062
I20260808 11:44:42.200177 25619 tablet_replica.cc:330] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: stopping tablet replica
I20260808 11:44:42.200222 25401 tablet_service.cc:1514] Processing DeleteTablet for tablet cef7b2dee74249aca79b7261b2b8321e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:42 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:57540
I20260808 11:44:42.200452 25619 raft_consensus.cc:2229] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:44:42.200511 25621 tablet_replica.cc:330] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: stopping tablet replica
I20260808 11:44:42.200712 25621 raft_consensus.cc:2229] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:42.200793 25619 raft_consensus.cc:2258] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:42.200943 25621 raft_consensus.cc:2258] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:42.201205 25618 tablet_replica.cc:330] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: stopping tablet replica
I20260808 11:44:42.201116 24822 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 11:44:42.201403 25618 raft_consensus.cc:2229] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:42.201615 25618 raft_consensus.cc:2258] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 11:44:42.201783 24822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:44:42.202654 25621 ts_tablet_manager.cc:1905] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:42.202883 25619 ts_tablet_manager.cc:1905] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:42.207322 25618 ts_tablet_manager.cc:1905] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:42.209009 25621 ts_tablet_manager.cc:1918] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:42.209051 25017 tablet_service.cc:1467] Processing CreateTablet for tablet c575d9ae53274595a5cedbb6afd867cb (DEFAULT_TABLE table=test-table [id=6a607ade6b1e42ce96c58a9254cecf37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:42.209245 25621 log.cc:1196] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/cef7b2dee74249aca79b7261b2b8321e
I20260808 11:44:42.209528 25017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c575d9ae53274595a5cedbb6afd867cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:42.209703 25621 ts_tablet_manager.cc:1939] T cef7b2dee74249aca79b7261b2b8321e P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting consensus metadata
I20260808 11:44:42.210124 25202 tablet_service.cc:1467] Processing CreateTablet for tablet c575d9ae53274595a5cedbb6afd867cb (DEFAULT_TABLE table=test-table [id=6a607ade6b1e42ce96c58a9254cecf37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:42.210539 25202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c575d9ae53274595a5cedbb6afd867cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:42.213688 25618 ts_tablet_manager.cc:1918] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:42.213917 25618 log.cc:1196] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/cef7b2dee74249aca79b7261b2b8321e
I20260808 11:44:42.214356 25618 ts_tablet_manager.cc:1939] T cef7b2dee74249aca79b7261b2b8321e P cbbbb6b12fc44c4285a5d7219e688beb: Deleting consensus metadata
I20260808 11:44:42.215051 25401 tablet_service.cc:1467] Processing CreateTablet for tablet c575d9ae53274595a5cedbb6afd867cb (DEFAULT_TABLE table=test-table [id=6a607ade6b1e42ce96c58a9254cecf37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:44:42.215461 25401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c575d9ae53274595a5cedbb6afd867cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:42.215535 24801 catalog_manager.cc:4841] TS cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733): tablet cef7b2dee74249aca79b7261b2b8321e (table test-table [id=cd7d43c2254f442990b6f76ed601ccad]) successfully deleted
I20260808 11:44:42.216634 25535 tablet_bootstrap.cc:492] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: Bootstrap starting.
I20260808 11:44:42.218005 25535 tablet_bootstrap.cc:654] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:42.218952 24801 catalog_manager.cc:4841] TS 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101): tablet cef7b2dee74249aca79b7261b2b8321e (table test-table [id=cd7d43c2254f442990b6f76ed601ccad]) successfully deleted
I20260808 11:44:42.219913 25535 tablet_bootstrap.cc:492] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: No bootstrap required, opened a new log
I20260808 11:44:42.220109 25535 ts_tablet_manager.cc:1397] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 11:44:42.220719 25536 tablet_bootstrap.cc:492] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: Bootstrap starting.
I20260808 11:44:42.220888 25535 raft_consensus.cc:348] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.221051 25535 raft_consensus.cc:374] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:42.221095 25535 raft_consensus.cc:729] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4cb1cfaa864c988999beb92695e272, State: Initialized, Role: FOLLOWER
I20260808 11:44:42.221313 25535 consensus_queue.cc:260] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.221741 25535 ts_tablet_manager.cc:1428] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 11:44:42.221958 25536 tablet_bootstrap.cc:654] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:42.224079 25536 tablet_bootstrap.cc:492] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: No bootstrap required, opened a new log
I20260808 11:44:42.224251 25536 ts_tablet_manager.cc:1397] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 11:44:42.224728 25619 ts_tablet_manager.cc:1918] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:44:42.225040 25536 raft_consensus.cc:348] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.225272 25619 log.cc:1196] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/cef7b2dee74249aca79b7261b2b8321e
I20260808 11:44:42.225286 25536 raft_consensus.cc:374] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:42.225374 25536 raft_consensus.cc:729] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbbb6b12fc44c4285a5d7219e688beb, State: Initialized, Role: FOLLOWER
I20260808 11:44:42.225503 25534 tablet_bootstrap.cc:492] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: Bootstrap starting.
I20260808 11:44:42.225745 25619 ts_tablet_manager.cc:1939] T cef7b2dee74249aca79b7261b2b8321e P fd4cb1cfaa864c988999beb92695e272: Deleting consensus metadata
I20260808 11:44:42.226135 25536 consensus_queue.cc:260] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.226902 25536 ts_tablet_manager.cc:1428] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 11:44:42.226940 24804 catalog_manager.cc:4841] TS fd4cb1cfaa864c988999beb92695e272 (127.13.158.66:41331): tablet cef7b2dee74249aca79b7261b2b8321e (table test-table [id=cd7d43c2254f442990b6f76ed601ccad]) successfully deleted
I20260808 11:44:42.227533 25534 tablet_bootstrap.cc:654] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:42.229616 25534 tablet_bootstrap.cc:492] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: No bootstrap required, opened a new log
I20260808 11:44:42.229813 25534 ts_tablet_manager.cc:1397] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260808 11:44:42.230504 25534 raft_consensus.cc:348] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.230778 25534 raft_consensus.cc:374] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:42.230885 25534 raft_consensus.cc:729] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85290b6a4c444b2f83e0e91d67a7ac2a, State: Initialized, Role: FOLLOWER
I20260808 11:44:42.231120 25534 consensus_queue.cc:260] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.231571 25534 ts_tablet_manager.cc:1428] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 11:44:42.457998 25543 raft_consensus.cc:482] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:44:42.458168 25543 raft_consensus.cc:504] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.458921 25543 leader_election.cc:288] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd4cb1cfaa864c988999beb92695e272 (127.13.158.66:41331), cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:42.459409 25236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c575d9ae53274595a5cedbb6afd867cb" candidate_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd4cb1cfaa864c988999beb92695e272" is_pre_election: true
I20260808 11:44:42.459575 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c575d9ae53274595a5cedbb6afd867cb" candidate_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" is_pre_election: true
I20260808 11:44:42.459690 25236 raft_consensus.cc:2454] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85290b6a4c444b2f83e0e91d67a7ac2a in term 0.
I20260808 11:44:42.459817 25428 raft_consensus.cc:2454] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85290b6a4c444b2f83e0e91d67a7ac2a in term 0.
I20260808 11:44:42.460237 24961 leader_election.cc:302] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85290b6a4c444b2f83e0e91d67a7ac2a, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:42.460495 25543 raft_consensus.cc:2781] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:44:42.460649 25543 raft_consensus.cc:482] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:44:42.460772 25543 raft_consensus.cc:3037] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:42.463392 25543 raft_consensus.cc:504] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.464159 25543 leader_election.cc:288] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [CANDIDATE]: Term 1 election: Requested vote from peers fd4cb1cfaa864c988999beb92695e272 (127.13.158.66:41331), cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733)
I20260808 11:44:42.464536 25236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c575d9ae53274595a5cedbb6afd867cb" candidate_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd4cb1cfaa864c988999beb92695e272"
I20260808 11:44:42.464663 25428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c575d9ae53274595a5cedbb6afd867cb" candidate_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbbbb6b12fc44c4285a5d7219e688beb"
I20260808 11:44:42.464771 25236 raft_consensus.cc:3037] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:42.464867 25428 raft_consensus.cc:3037] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:42.466843 25236 raft_consensus.cc:2454] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85290b6a4c444b2f83e0e91d67a7ac2a in term 1.
I20260808 11:44:42.467237 24961 leader_election.cc:302] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85290b6a4c444b2f83e0e91d67a7ac2a, fd4cb1cfaa864c988999beb92695e272; no voters:
I20260808 11:44:42.467511 25543 raft_consensus.cc:2781] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:42.467825 25428 raft_consensus.cc:2454] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85290b6a4c444b2f83e0e91d67a7ac2a in term 1.
I20260808 11:44:42.467993 25543 raft_consensus.cc:686] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 LEADER]: Becoming Leader. State: Replica: 85290b6a4c444b2f83e0e91d67a7ac2a, State: Running, Role: LEADER
I20260808 11:44:42.468552 25543 consensus_queue.cc:237] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } }
I20260808 11:44:42.472549 24822 catalog_manager.cc:5495] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65). New cstate: current_term: 1 leader_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85290b6a4c444b2f83e0e91d67a7ac2a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 44101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 } health_report { overall_health: UNKNOWN } } }
I20260808 11:44:42.524168 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.518846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42660 (local address 127.13.158.65:44101)
0808 11:44:42.519457 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:42.519462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:42.519480 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:44:42.519560 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:42.519564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:42.519644 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:42.519782 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:42.520106 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.520861 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.522047 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.522449 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.522531 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.522923 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:42.523650 (+ 727us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:42.523758 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:42.523844 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 11:44:42.523905 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":365,"threads_started":1}
I20260808 11:44:42.538568 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.533632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:54535 (local address 127.13.158.66:41331)
0808 11:44:42.533867 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:42.533876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:42.534406 (+ 530us) server_negotiation.cc:407] Connection header received
0808 11:44:42.534545 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:42.534548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:42.534625 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:42.534738 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:42.535167 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.535867 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.537151 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.537547 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.537638 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.538002 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:42.538101 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:42.538175 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:42.538297 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 11:44:42.538368 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:44:42.538766 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.533633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41331 (local address 127.13.158.65:54535)
0808 11:44:42.534306 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:42.534323 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:42.534438 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:42.534767 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:42.534770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:42.534780 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:44:42.535032 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:42.535039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.535980 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.535984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:42.537030 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:42.537041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.537674 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.537679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:42.537817 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.537866 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:44:42.538191 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:44:42.538199 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:44:42.538259 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 11:44:42.538409 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":420,"threads_started":1}
I20260808 11:44:42.546880 25428 raft_consensus.cc:1261] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Refusing update from remote peer 85290b6a4c444b2f83e0e91d67a7ac2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:42.547165 25236 raft_consensus.cc:1261] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Refusing update from remote peer 85290b6a4c444b2f83e0e91d67a7ac2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:44:42.547480 25650 consensus_queue.cc:1035] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbbbb6b12fc44c4285a5d7219e688beb" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 45733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:42.547868 25543 consensus_queue.cc:1035] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd4cb1cfaa864c988999beb92695e272" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:44:42.573532 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.568575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:50811 (local address 127.13.158.65:44101)
0808 11:44:42.568737 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:42.568742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:42.568988 (+ 246us) server_negotiation.cc:407] Connection header received
0808 11:44:42.569115 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:42.569118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:42.569172 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:42.569264 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:44:42.569665 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.570493 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.572198 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.572595 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.572698 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.573008 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:44:42.573096 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:44:42.573164 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:42.573280 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 11:44:42.573337 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 11:44:42.573578 25667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.568237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:44101 (local address 127.13.158.66:50811)
0808 11:44:42.568925 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:42.568940 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:42.569035 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:42.569281 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:42.569284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:42.569293 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:44:42.569568 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:42.569577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.570627 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.570632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:42.572076 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:42.572089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.572744 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.572749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:42.572852 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.572886 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:44:42.573184 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:44:42.573193 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:44:42.573253 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 11:44:42.573344 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":425,"threads_started":1}
I20260808 11:44:42.599123 24822 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:42.599318 24822 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:44:42.602246 24822 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a4b001fae7e4a1b8cf763a9cf0685db: Sending DeleteTablet for 3 replicas of tablet c575d9ae53274595a5cedbb6afd867cb
I20260808 11:44:42.603070 25202 tablet_service.cc:1514] Processing DeleteTablet for tablet c575d9ae53274595a5cedbb6afd867cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:42 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:48062
I20260808 11:44:42.603394 25401 tablet_service.cc:1514] Processing DeleteTablet for tablet c575d9ae53274595a5cedbb6afd867cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:42 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:57540
I20260808 11:44:42.603461 25619 tablet_replica.cc:330] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: stopping tablet replica
I20260808 11:44:42.603642 25619 raft_consensus.cc:2229] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:42.603672 25618 tablet_replica.cc:330] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: stopping tablet replica
I20260808 11:44:42.603842 25618 raft_consensus.cc:2229] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:44:42.603847 25619 raft_consensus.cc:2258] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:42.604046 25618 raft_consensus.cc:2258] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:42.604141 25017 tablet_service.cc:1514] Processing DeleteTablet for tablet c575d9ae53274595a5cedbb6afd867cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:44:42 UTC) from {username='oryx', principal='oryx/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:56734
I20260808 11:44:42.604436 25621 tablet_replica.cc:330] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: stopping tablet replica
I20260808 11:44:42.604483 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 24937
I20260808 11:44:42.604630 25621 raft_consensus.cc:2229] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:44:42.605155 25621 raft_consensus.cc:2258] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:44:42.607239 25619 ts_tablet_manager.cc:1905] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:42.607472 25618 ts_tablet_manager.cc:1905] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:42.614331 25618 ts_tablet_manager.cc:1918] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:44:42.615875 25621 ts_tablet_manager.cc:1905] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:44:42.617146 25619 ts_tablet_manager.cc:1918] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:44:42.617268 25619 log.cc:1196] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/c575d9ae53274595a5cedbb6afd867cb
I20260808 11:44:42.617574 25619 ts_tablet_manager.cc:1939] T c575d9ae53274595a5cedbb6afd867cb P fd4cb1cfaa864c988999beb92695e272: Deleting consensus metadata
I20260808 11:44:42.618595 24804 catalog_manager.cc:4841] TS fd4cb1cfaa864c988999beb92695e272 (127.13.158.66:41331): tablet c575d9ae53274595a5cedbb6afd867cb (table test-table [id=6a607ade6b1e42ce96c58a9254cecf37]) successfully deleted
I20260808 11:44:42.618993 25618 log.cc:1196] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/c575d9ae53274595a5cedbb6afd867cb
I20260808 11:44:42.619499 25618 ts_tablet_manager.cc:1939] T c575d9ae53274595a5cedbb6afd867cb P cbbbb6b12fc44c4285a5d7219e688beb: Deleting consensus metadata
I20260808 11:44:42.620640 24801 catalog_manager.cc:4841] TS cbbbb6b12fc44c4285a5d7219e688beb (127.13.158.67:45733): tablet c575d9ae53274595a5cedbb6afd867cb (table test-table [id=6a607ade6b1e42ce96c58a9254cecf37]) successfully deleted
I20260808 11:44:42.624033 25621 ts_tablet_manager.cc:1918] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:44:42.624220 25621 log.cc:1196] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/c575d9ae53274595a5cedbb6afd867cb
I20260808 11:44:42.624668 25621 ts_tablet_manager.cc:1939] T c575d9ae53274595a5cedbb6afd867cb P 85290b6a4c444b2f83e0e91d67a7ac2a: Deleting consensus metadata
I20260808 11:44:42.625892 24801 catalog_manager.cc:4841] TS 85290b6a4c444b2f83e0e91d67a7ac2a (127.13.158.65:44101): tablet c575d9ae53274595a5cedbb6afd867cb (table test-table [id=6a607ade6b1e42ce96c58a9254cecf37]) successfully deleted
Aug 08 11:44:42 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.65@KRBTEST.COM
I20260808 11:44:42.636051 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.615414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.13.158.65:44101)
0808 11:44:42.615583 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:42.615590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:42.618393 (+ 2803us) server_negotiation.cc:407] Connection header received
0808 11:44:42.618493 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:42.618498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:42.618592 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:42.618732 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:42.620636 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.621804 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.623730 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.624484 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.624619 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.628965 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:42.628979 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:42.628999 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:42.629038 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:42.633121 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:42.633769 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:42.633778 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:42.633782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:42.633958 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:42.634403 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:42.634408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:42.634411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:42.634871 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:42.635223 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:42.635413 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 11:44:42.635628 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:44:42.636914 25107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:42.743047 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 24937
I20260808 11:44:42.769837 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 25118
Aug 08 11:44:42 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.66@KRBTEST.COM
I20260808 11:44:42.785039 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.770773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.13.158.66:41331)
0808 11:44:42.770960 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:42.770966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:42.770984 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:44:42.771088 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:42.771092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:42.771157 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:42.771265 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:42.772547 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.773251 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.774478 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.774972 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.775136 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.779371 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:42.779385 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:42.779399 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:42.779437 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:42.782211 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:42.782893 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:42.782901 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:42.782903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:42.783069 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:42.783469 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:42.783473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:42.783476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:42.783827 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:42.784152 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:42.784475 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 11:44:42.784700 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:44:42.785907 25289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:42.931173 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 25118
I20260808 11:44:42.966898 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 25301
Aug 08 11:44:42 dist-test-slave-39l8 krb5kdc[24757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.158.67@KRBTEST.COM
I20260808 11:44:42.982988 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:42.967832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60312 (local address 127.13.158.67:45733)
0808 11:44:42.968351 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:42.968356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:42.968370 (+ 14us) server_negotiation.cc:407] Connection header received
0808 11:44:42.968448 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:42.968456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:42.968534 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:42.968663 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:42.970103 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.970975 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.972195 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:42.972606 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:42.972755 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:42.977182 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:42.977197 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:42.977220 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:42.977255 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:42.980090 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:42.980756 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:42.980764 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:42.980766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:42.980894 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:42.981294 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:42.981300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:42.981303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:42.981706 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:42.982003 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:42.982315 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 11:44:42.982545 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":268,"threads_started":1}
I20260808 11:44:42.983752 25496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:43.079416 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 25301
I20260808 11:44:43.104732 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 24777
I20260808 11:44:43.106060 24889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:43.222901 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 24777
2026-08-08T11:44:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6358 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:43 dist-test-slave-39l8 krb5kdc[25684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:43 dist-test-slave-39l8 krb5kdc[25684](info): set up 2 sockets
Aug 08 11:44:43 dist-test-slave-39l8 krb5kdc[25684](info): commencing operation
krb5kdc: starting...
W20260808 11:44:44.507443 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:44 dist-test-slave-39l8 krb5kdc[25684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:44:44.741686 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:38563
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:38563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:45.066902 25702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:45.067191 25702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:45.067384 25702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:45.074484 25702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:45.074586 25702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:45.074674 25702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:45.074801 25702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:45.074857 25702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:45.084610 25702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:38563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:38563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:45.085770 25702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:45.087059 25702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:45.098670 25708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:45.098584 25709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:45.098634 25712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:45.099551 25702 server_base.cc:1034] running on GCE node
I20260808 11:44:45.100324 25702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:45.101650 25702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:45.102865 25702 hybrid_clock.cc:648] HybridClock initialized: now 1786189485102822 us; error 53 us; skew 500 ppm
I20260808 11:44:45.103248 25702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T11:44:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1936 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:44:45 dist-test-slave-39l8 krb5kdc[25721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:44:45 dist-test-slave-39l8 krb5kdc[25721](info): set up 2 sockets
Aug 08 11:44:45 dist-test-slave-39l8 krb5kdc[25721](info): commencing operation
krb5kdc: starting...
W20260808 11:44:46.430644 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:44:46 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:44:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:44:46.668478 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43027
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:43027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:46.993361 25738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:46.993690 25738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:46.993858 25738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:47.000834 25738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:47.000973 25738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:47.001087 25738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:47.001186 25738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:47.001260 25738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:47.011104 25738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:43027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:47.012504 25738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:47.013926 25738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:47.025362 25746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:47.025472 25749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:47.025501 25751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:47.026242 25738 server_base.cc:1034] running on GCE node
I20260808 11:44:47.027052 25738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:47.028270 25738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:47.029472 25738 hybrid_clock.cc:648] HybridClock initialized: now 1786189487029432 us; error 32 us; skew 500 ppm
I20260808 11:44:47.029757 25738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:47 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189487, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:47.037715 25738 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:44:47.039170 25738 webserver.cc:460] Webserver started at http://127.13.158.126:41945/ using document root <none> and password file <none>
I20260808 11:44:47.039790 25738 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:47.039884 25738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:47.040135 25738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:47.042987 25738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "de3c5aeea56b4183bafc5c6d8dfc9053"
format_stamp: "Formatted at 2026-08-08 11:44:47 on dist-test-slave-39l8"
server_key: "a76ffc031e674f2ada500238ca5564f7"
server_key_iv: "54160981d4ac3157d86a18ee2ee2dd0e"
server_key_version: "encryptionkey@0"
I20260808 11:44:47.043700 25738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "de3c5aeea56b4183bafc5c6d8dfc9053"
format_stamp: "Formatted at 2026-08-08 11:44:47 on dist-test-slave-39l8"
server_key: "a76ffc031e674f2ada500238ca5564f7"
server_key_iv: "54160981d4ac3157d86a18ee2ee2dd0e"
server_key_version: "encryptionkey@0"
I20260808 11:44:47.048447 25738 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:44:47.051543 25759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:47.052809 25738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:47.052983 25738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "de3c5aeea56b4183bafc5c6d8dfc9053"
format_stamp: "Formatted at 2026-08-08 11:44:47 on dist-test-slave-39l8"
server_key: "a76ffc031e674f2ada500238ca5564f7"
server_key_iv: "54160981d4ac3157d86a18ee2ee2dd0e"
server_key_version: "encryptionkey@0"
I20260808 11:44:47.053179 25738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:47.077965 25738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:47.082427 25738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:47.082810 25738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:47.101153 25738 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:43027
I20260808 11:44:47.101235 25834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:43027 every 8 connection(s)
I20260808 11:44:47.102478 25738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:47.106243 25835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:44:47.111752 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 25738
I20260808 11:44:47.111969 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:44:47.112891 13945 external_mini_cluster.cc:1428] Setting key 8d45d629344d6500f07a2812e07f4edd
I20260808 11:44:47.117672 25835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Bootstrap starting.
I20260808 11:44:47.121341 25835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Neither blocks nor log segments found. Creating new log.
I20260808 11:44:47.122597 25835 log.cc:824] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:47.125857 25835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: No bootstrap required, opened a new log
Aug 08 11:44:47 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:47.134295 25835 raft_consensus.cc:348] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } }
I20260808 11:44:47.134660 25835 raft_consensus.cc:374] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:44:47.134809 25835 raft_consensus.cc:729] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3c5aeea56b4183bafc5c6d8dfc9053, State: Initialized, Role: FOLLOWER
I20260808 11:44:47.135478 25835 consensus_queue.cc:260] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } }
I20260808 11:44:47.135743 25835 raft_consensus.cc:388] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:47.135877 25835 raft_consensus.cc:482] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:47.136099 25835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:44:47.137537 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:47.115090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.13.158.126:43027)
0808 11:44:47.115939 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:47.115949 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:47.115993 (+ 44us) server_negotiation.cc:407] Connection header received
0808 11:44:47.117662 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:47.117683 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:47.118093 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:47.118383 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:47.120291 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:47.121567 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:47.123407 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:47.124287 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:47.124412 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:47.129094 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:47.129101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:47.129114 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:47.129147 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:47.132804 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:47.133631 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:47.133639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:47.133655 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:47.133841 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:47.134287 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:47.134292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:47.134294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:47.135072 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:47.135430 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:47.135926 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 11:44:47.136406 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":436,"threads_started":1}
I20260808 11:44:47.138631 25835 raft_consensus.cc:504] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } }
I20260808 11:44:47.139302 25835 leader_election.cc:302] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de3c5aeea56b4183bafc5c6d8dfc9053; no voters:
I20260808 11:44:47.139981 25835 leader_election.cc:288] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:44:47.140086 25850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:44:47.141388 25850 raft_consensus.cc:686] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 1 LEADER]: Becoming Leader. State: Replica: de3c5aeea56b4183bafc5c6d8dfc9053, State: Running, Role: LEADER
I20260808 11:44:47.141997 25850 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } }
I20260808 11:44:47.142509 25835 sys_catalog.cc:564] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:47.145145 25854 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3c5aeea56b4183bafc5c6d8dfc9053. Latest consensus state: current_term: 1 leader_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } } }
I20260808 11:44:47.145589 25854 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:47.145684 25851 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } } }
I20260808 11:44:47.146126 25851 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:47.152486 25861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:47.158739 25861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:47.168982 25861 catalog_manager.cc:1347] Generated new cluster ID: ea6c7b34fa34447ea1263ed16ad429cd
I20260808 11:44:47.169097 25861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:47.193778 25861 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:44:47.195885 25861 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:47.209560 25861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Generated new TSK 0
I20260808 11:44:47.210695 25861 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:44:47.288564 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43027
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:47.636982 25888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:47.637312 25888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:47.637491 25888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:47.647452 25888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:47.647640 25888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:47.647879 25888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:47.666090 25888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:47.668036 25888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:47.669958 25888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:47.683483 25894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:47.683499 25895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:47.683774 25897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:47.684559 25888 server_base.cc:1034] running on GCE node
I20260808 11:44:47.685205 25888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:47.686136 25888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:47.687367 25888 hybrid_clock.cc:648] HybridClock initialized: now 1786189487687317 us; error 58 us; skew 500 ppm
I20260808 11:44:47.687719 25888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:47 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189487, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:47.691150 25888 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:44:47.692664 25888 webserver.cc:460] Webserver started at http://127.13.158.65:32865/ using document root <none> and password file <none>
I20260808 11:44:47.693382 25888 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:47.693542 25888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:47.693871 25888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:47.696916 25888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "fd872afc0afd4daf893c293b7a7799cd"
format_stamp: "Formatted at 2026-08-08 11:44:47 on dist-test-slave-39l8"
server_key: "fd479db6b97d74aaa8816592185ff35d"
server_key_iv: "82346660a361b6aa063957bf4abc2403"
server_key_version: "encryptionkey@0"
I20260808 11:44:47.697759 25888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "fd872afc0afd4daf893c293b7a7799cd"
format_stamp: "Formatted at 2026-08-08 11:44:47 on dist-test-slave-39l8"
server_key: "fd479db6b97d74aaa8816592185ff35d"
server_key_iv: "82346660a361b6aa063957bf4abc2403"
server_key_version: "encryptionkey@0"
I20260808 11:44:47.703239 25888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:44:47.706763 25906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:47.708106 25888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:44:47.708302 25888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "fd872afc0afd4daf893c293b7a7799cd"
format_stamp: "Formatted at 2026-08-08 11:44:47 on dist-test-slave-39l8"
server_key: "fd479db6b97d74aaa8816592185ff35d"
server_key_iv: "82346660a361b6aa063957bf4abc2403"
server_key_version: "encryptionkey@0"
I20260808 11:44:47.708535 25888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:47.726300 25888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:47.731498 25888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:47.731822 25888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:47.732851 25888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:47.734263 25888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:47.734376 25888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:47.734536 25888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:47.734627 25888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:47.769250 25888 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:46759
I20260808 11:44:47.769315 26057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:46759 every 8 connection(s)
I20260808 11:44:47.771095 25888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:47.772815 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 25888
I20260808 11:44:47.773033 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:44:47.774011 13945 external_mini_cluster.cc:1428] Setting key d76db79c93575e8082ab4fb83275d977
Aug 08 11:44:47 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189487, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:47.791690 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:47.773922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:50611 (local address 127.13.158.126:43027)
0808 11:44:47.774422 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:47.774428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:47.775657 (+ 1229us) server_negotiation.cc:407] Connection header received
0808 11:44:47.777513 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:47.777518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:47.777611 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:47.777734 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:47.779261 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:47.780052 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:47.781581 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:47.782009 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:47.782155 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:47.785950 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:47.785955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:47.785958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:47.785992 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:47.788548 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:47.789304 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:47.789309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:47.789314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:47.789436 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:47.789890 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:47.789895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:47.789898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:47.790161 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:47.790334 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:47.791088 (+ 754us) server_negotiation.cc:299] Negotiation successful
0808 11:44:47.791325 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":290,"threads_started":1}
I20260808 11:44:47.792682 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:47.774673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:50611)
0808 11:44:47.775503 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:47.775540 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:47.777252 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:47.777972 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:47.777980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:47.778476 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:47.779091 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:47.779108 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:47.780224 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:47.780228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:47.781437 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:47.781450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:47.782254 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:47.782263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:47.782494 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:47.783236 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:47.783259 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:47.785710 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:47.788725 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:47.788734 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:47.788749 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:47.789177 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:47.789549 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:47.789554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:47.789558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:47.789767 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:47.790296 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:47.790303 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:47.790639 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 11:44:47.790980 (+ 341us) client_negotiation.cc:240] Negotiation successful
0808 11:44:47.791490 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":313,"threads_started":1}
I20260808 11:44:47.794294 26058 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43027
I20260808 11:44:47.794600 26058 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:47.795358 26058 heartbeater.cc:511] Master 127.13.158.126:43027 requested a full tablet report, sending...
I20260808 11:44:47.797407 25784 ts_manager.cc:194] Registered new tserver with Master: fd872afc0afd4daf893c293b7a7799cd (127.13.158.65:46759)
I20260808 11:44:47.798943 25784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:50611
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:44:47.831684 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43027
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:48.163643 26065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:48.163897 26065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:48.163990 26065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:48.170992 26065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:48.171087 26065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:48.171293 26065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:48.180545 26065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:48.181811 26065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:48.183063 26065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:48.195691 26070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:48.195784 26074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:48.195820 26072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:48.197211 26065 server_base.cc:1034] running on GCE node
I20260808 11:44:48.197784 26065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:48.198634 26065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:48.199878 26065 hybrid_clock.cc:648] HybridClock initialized: now 1786189488199827 us; error 46 us; skew 500 ppm
I20260808 11:44:48.200168 26065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:48 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189488, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:48.203295 26065 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:44:48.204698 26065 webserver.cc:460] Webserver started at http://127.13.158.66:41385/ using document root <none> and password file <none>
I20260808 11:44:48.205401 26065 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:48.205543 26065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:48.205886 26065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:48.208699 26065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "ccb0de9f017b40b58ca626bc5db2ed31"
format_stamp: "Formatted at 2026-08-08 11:44:48 on dist-test-slave-39l8"
server_key: "8ada4531892f4bc4f14f865fce22b75f"
server_key_iv: "089e31aeb16ecf6c0a007ba2b4d20a97"
server_key_version: "encryptionkey@0"
I20260808 11:44:48.209496 26065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "ccb0de9f017b40b58ca626bc5db2ed31"
format_stamp: "Formatted at 2026-08-08 11:44:48 on dist-test-slave-39l8"
server_key: "8ada4531892f4bc4f14f865fce22b75f"
server_key_iv: "089e31aeb16ecf6c0a007ba2b4d20a97"
server_key_version: "encryptionkey@0"
I20260808 11:44:48.214617 26065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 11:44:48.217975 26082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:48.219246 26065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:44:48.219419 26065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "ccb0de9f017b40b58ca626bc5db2ed31"
format_stamp: "Formatted at 2026-08-08 11:44:48 on dist-test-slave-39l8"
server_key: "8ada4531892f4bc4f14f865fce22b75f"
server_key_iv: "089e31aeb16ecf6c0a007ba2b4d20a97"
server_key_version: "encryptionkey@0"
I20260808 11:44:48.219614 26065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:48.245852 26065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:48.249847 26065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:48.250145 26065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:48.251188 26065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:48.252549 26065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:48.252657 26065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:48.252820 26065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:48.252897 26065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:48.286579 26065 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:37091
I20260808 11:44:48.286619 26253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:37091 every 8 connection(s)
I20260808 11:44:48.287854 26065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:48.296749 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 26065
I20260808 11:44:48.296926 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:44:48.297472 13945 external_mini_cluster.cc:1428] Setting key a0f06f1ba30561eedb65ac75e4089d75
Aug 08 11:44:48 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189488, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:48.308054 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:48.290315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:51765 (local address 127.13.158.126:43027)
0808 11:44:48.290486 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:48.290490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:48.291630 (+ 1140us) server_negotiation.cc:407] Connection header received
0808 11:44:48.293442 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:48.293450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:48.293521 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:48.293599 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:48.295120 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.295799 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.297837 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.298387 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.298482 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:48.302376 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:48.302380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:48.302384 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:48.302413 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:48.305015 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:48.305747 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:48.305751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:48.305754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:48.305873 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:48.306317 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:48.306327 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:48.306329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:48.306595 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:48.306816 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:48.307477 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 11:44:48.307704 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 11:44:48.308938 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:48.290760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:51765)
0808 11:44:48.291453 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:48.291494 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:48.293148 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:48.293862 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:48.293869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:48.294356 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:48.294972 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:48.294991 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.295929 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.295932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:48.297696 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:48.297719 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.298607 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.298614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:48.298857 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:48.299585 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:48.299614 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:48.302148 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:48.305157 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:48.305166 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:48.305181 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:48.305601 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:48.305993 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:48.305997 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:48.306001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:48.306183 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:48.306745 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:48.306752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:48.307071 (+ 319us) client_negotiation.cc:769] Sending connection context
0808 11:44:48.307330 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 11:44:48.307872 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":264,"threads_started":1}
I20260808 11:44:48.310286 26254 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43027
I20260808 11:44:48.310631 26254 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:48.311501 26254 heartbeater.cc:511] Master 127.13.158.126:43027 requested a full tablet report, sending...
I20260808 11:44:48.313118 25784 ts_manager.cc:194] Registered new tserver with Master: ccb0de9f017b40b58ca626bc5db2ed31 (127.13.158.66:37091)
I20260808 11:44:48.313992 25784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:51765
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:44:48.354414 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43027
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:48.688839 26263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:48.689188 26263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:48.689291 26263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:48.696394 26263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:48.696493 26263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:48.696614 26263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:48.706204 26263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:48.707453 26263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:48.708747 26263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:48.721961 26275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:48.721983 26276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:48.722311 26278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:48.722893 26263 server_base.cc:1034] running on GCE node
I20260808 11:44:48.723552 26263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:48.724540 26263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:48.725775 26263 hybrid_clock.cc:648] HybridClock initialized: now 1786189488725732 us; error 53 us; skew 500 ppm
I20260808 11:44:48.726070 26263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:48 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189488, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:48.729301 26263 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:44:48.730577 26263 webserver.cc:460] Webserver started at http://127.13.158.67:39865/ using document root <none> and password file <none>
I20260808 11:44:48.731251 26263 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:48.731346 26263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:48.731583 26263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:44:48.734352 26263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "7f18b6b67cc34e0fa3cf2f424427c41a"
format_stamp: "Formatted at 2026-08-08 11:44:48 on dist-test-slave-39l8"
server_key: "c3b3dee21f58ef0424a1d1262839fec8"
server_key_iv: "1e82b724633b06dc4f6cafc3380ffeb8"
server_key_version: "encryptionkey@0"
I20260808 11:44:48.735143 26263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "7f18b6b67cc34e0fa3cf2f424427c41a"
format_stamp: "Formatted at 2026-08-08 11:44:48 on dist-test-slave-39l8"
server_key: "c3b3dee21f58ef0424a1d1262839fec8"
server_key_iv: "1e82b724633b06dc4f6cafc3380ffeb8"
server_key_version: "encryptionkey@0"
I20260808 11:44:48.739990 26263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:44:48.743275 26288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:48.744580 26263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:44:48.744717 26263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "7f18b6b67cc34e0fa3cf2f424427c41a"
format_stamp: "Formatted at 2026-08-08 11:44:48 on dist-test-slave-39l8"
server_key: "c3b3dee21f58ef0424a1d1262839fec8"
server_key_iv: "1e82b724633b06dc4f6cafc3380ffeb8"
server_key_version: "encryptionkey@0"
I20260808 11:44:48.744889 26263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:48.761981 26263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:48.765770 26263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:48.766010 26263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:48.766916 26263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:48.768155 26263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:48.768227 26263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:48.768311 26263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:48.768354 26263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:48.802017 26058 heartbeater.cc:503] Master 127.13.158.126:43027 was elected leader, sending a full tablet report...
I20260808 11:44:48.803078 26263 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:38459
I20260808 11:44:48.803117 26455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:38459 every 8 connection(s)
I20260808 11:44:48.804385 26263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:48.807740 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 26263
I20260808 11:44:48.807941 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:44:48.808410 13945 external_mini_cluster.cc:1428] Setting key e999f4c83572c52e0e8bfb0c0213d4e2
Aug 08 11:44:48 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189488, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:44:48.824465 26060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:48.806961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:60285 (local address 127.13.158.126:43027)
0808 11:44:48.807163 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:48.807167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:48.808483 (+ 1316us) server_negotiation.cc:407] Connection header received
0808 11:44:48.810677 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:48.810682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:48.810823 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:48.810957 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:48.812604 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.813433 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.815022 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.815404 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.815538 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:48.818797 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:48.818801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:48.818804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:48.818835 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:48.821290 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:48.822007 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:48.822014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:48.822019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:48.822169 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:48.822580 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:48.822586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:48.822589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:48.822980 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:48.823157 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:48.823895 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 11:44:48.824114 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:44:48.825390 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:48.807436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:60285)
0808 11:44:48.808294 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:48.808348 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:48.810378 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:48.811233 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:48.811241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:48.811762 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:48.812412 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:48.812430 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.813602 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.813614 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:48.814887 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:48.814899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.815630 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.815639 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:48.815868 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:48.816515 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:48.816542 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:48.818568 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:44:48.821431 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:48.821439 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:48.821454 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:48.821888 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:48.822281 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:44:48.822286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:44:48.822289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:44:48.822467 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:44:48.823123 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:44:48.823130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:44:48.823515 (+ 385us) client_negotiation.cc:769] Sending connection context
0808 11:44:48.823804 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 11:44:48.824312 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":366,"threads_started":1}
I20260808 11:44:48.826607 26456 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43027
I20260808 11:44:48.826931 26456 heartbeater.cc:464] Registering TS with master...
I20260808 11:44:48.827631 26456 heartbeater.cc:511] Master 127.13.158.126:43027 requested a full tablet report, sending...
I20260808 11:44:48.829035 25784 ts_manager.cc:194] Registered new tserver with Master: 7f18b6b67cc34e0fa3cf2f424427c41a (127.13.158.67:38459)
I20260808 11:44:48.829818 25784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:60285
I20260808 11:44:48.835914 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab.
Entry for principal oryx/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab.
WARNING: no policy specified for oryx/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab.
Entry for principal oryx/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab.
WARNING: no policy specified for oryx/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab.
Entry for principal oryx/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab.
I20260808 11:44:48.993119 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 25888
Aug 08 11:44:49 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:44:49.009378 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:48.994419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.13.158.65:46759)
0808 11:44:48.995113 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:48.995120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:48.995181 (+ 61us) server_negotiation.cc:407] Connection header received
0808 11:44:48.995247 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:48.995254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:48.995490 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:48.995630 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:48.996986 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.997977 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.999252 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:48.999800 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:48.999897 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:49.003942 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:49.003953 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:49.003967 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:49.004000 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:49.006943 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:49.007529 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:49.007536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:49.007538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:49.007662 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:49.007971 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:49.007974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:49.007976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:49.008311 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:49.008541 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:49.008761 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 11:44:49.008986 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":385,"threads_started":1}
I20260808 11:44:49.010686 26051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:49.096154 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 25888
I20260808 11:44:49.119024 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 26065
Aug 08 11:44:49 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:44:49.134462 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:49.119855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.13.158.66:37091)
0808 11:44:49.120491 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:49.120498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:49.120525 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:44:49.120585 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:49.120591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:49.120800 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:49.120928 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:49.122117 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:49.123274 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:49.124567 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:49.125094 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:49.125199 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:49.129256 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:49.129268 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:49.129281 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:49.129313 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:49.132188 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:49.132772 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:49.132779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:49.132781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:49.132893 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:49.133204 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:49.133214 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:49.133216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:49.133527 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:49.133758 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:49.133967 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 11:44:49.134157 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":353,"threads_started":1}
I20260808 11:44:49.135807 26247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:49.217887 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 26065
I20260808 11:44:49.240224 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 26263
Aug 08 11:44:49 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:44:49.257148 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:49.241198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56932 (local address 127.13.158.67:38459)
0808 11:44:49.241797 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:49.241804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:49.241833 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:44:49.241898 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:49.241905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:49.242175 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:49.242375 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:49.243868 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:49.244822 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:49.246075 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:49.246626 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:49.246923 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:49.251422 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:49.251432 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:49.251445 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:49.251477 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:49.254324 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:49.255027 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:49.255034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:49.255036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:49.255156 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:49.255540 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:49.255544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:49.255546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:49.255872 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:49.256166 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:49.256521 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 11:44:49.256786 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":303,"threads_started":1}
I20260808 11:44:49.258558 26449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:49.339669 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 26263
I20260808 11:44:49.361497 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 25738
I20260808 11:44:49.362849 25829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:44:49.466195 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 25738
I20260808 11:44:49.493803 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43027
--webserver_interface=127.13.158.126
--webserver_port=41945
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:43027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:44:49.826478 26483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:49.826872 26483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:49.827042 26483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:49.834528 26483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:44:49.834674 26483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:49.834820 26483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:49.834926 26483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:44:49.835011 26483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:44:49.846113 26483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:43027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:49.847621 26483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:49.848941 26483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:49.860677 26495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:49.860620 26496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:49.860579 26498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:49.861603 26483 server_base.cc:1034] running on GCE node
I20260808 11:44:49.862375 26483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:49.863818 26483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:49.865047 26483 hybrid_clock.cc:648] HybridClock initialized: now 1786189489864999 us; error 60 us; skew 500 ppm
I20260808 11:44:49.865394 26483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:49 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189489, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:49.868692 26483 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:44:49.870070 26483 webserver.cc:460] Webserver started at http://127.13.158.126:41945/ using document root <none> and password file <none>
I20260808 11:44:49.870797 26483 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:49.870914 26483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:49.876219 26483 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:44:49.879002 26506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:49.880326 26483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:44:49.880465 26483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "de3c5aeea56b4183bafc5c6d8dfc9053"
format_stamp: "Formatted at 2026-08-08 11:44:47 on dist-test-slave-39l8"
server_key: "a76ffc031e674f2ada500238ca5564f7"
server_key_iv: "54160981d4ac3157d86a18ee2ee2dd0e"
server_key_version: "encryptionkey@0"
I20260808 11:44:49.881358 26483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:49.911044 26483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:49.915761 26483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:49.916006 26483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:49.934893 26483 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:43027
I20260808 11:44:49.934912 26585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:43027 every 8 connection(s)
I20260808 11:44:49.936445 26483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:44:49.939684 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 26483
I20260808 11:44:49.940512 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:46759
--local_ip_for_outbound_sockets=127.13.158.65
--tserver_master_addrs=127.13.158.126:43027
--webserver_port=32865
--webserver_interface=127.13.158.65
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.158.65
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:44:49.943423 26586 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:44:49.946794 26586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Bootstrap starting.
I20260808 11:44:49.965792 26586 log.cc:824] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Log is configured to *not* fsync() on all Append() calls
I20260808 11:44:49.974496 26586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:44:49.975085 26586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Bootstrap complete.
I20260808 11:44:49.983803 26586 raft_consensus.cc:348] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } }
I20260808 11:44:49.984675 26586 raft_consensus.cc:729] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3c5aeea56b4183bafc5c6d8dfc9053, State: Initialized, Role: FOLLOWER
I20260808 11:44:49.985296 26586 consensus_queue.cc:260] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } }
I20260808 11:44:49.985527 26586 raft_consensus.cc:388] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:44:49.985634 26586 raft_consensus.cc:482] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:44:49.985795 26586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:44:49.989369 26586 raft_consensus.cc:504] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } }
I20260808 11:44:49.989841 26586 leader_election.cc:302] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de3c5aeea56b4183bafc5c6d8dfc9053; no voters:
I20260808 11:44:49.990486 26586 leader_election.cc:288] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 11:44:49.990621 26591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:44:49.993034 26586 sys_catalog.cc:564] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:44:49.997815 26591 raft_consensus.cc:686] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [term 2 LEADER]: Becoming Leader. State: Replica: de3c5aeea56b4183bafc5c6d8dfc9053, State: Running, Role: LEADER
I20260808 11:44:49.998463 26591 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } }
I20260808 11:44:50.004108 26594 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3c5aeea56b4183bafc5c6d8dfc9053. Latest consensus state: current_term: 2 leader_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } } }
I20260808 11:44:50.004603 26594 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:50.005256 26592 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3c5aeea56b4183bafc5c6d8dfc9053" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43027 } } }
I20260808 11:44:50.005594 26592 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053 [sys.catalog]: This master's current role is: LEADER
I20260808 11:44:50.012125 26620 catalog_manager.cc:1259] Loaded cluster ID: ea6c7b34fa34447ea1263ed16ad429cd
I20260808 11:44:50.012290 26620 catalog_manager.cc:1552] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: loading cluster ID for follower catalog manager: success
I20260808 11:44:50.016108 26620 catalog_manager.cc:1574] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: acquiring CA information for follower catalog manager: success
I20260808 11:44:50.018453 26620 catalog_manager.cc:1602] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:44:50.018693 26622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:44:50.021118 26622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:44:50.022269 26622 catalog_manager.cc:1259] Loaded cluster ID: ea6c7b34fa34447ea1263ed16ad429cd
I20260808 11:44:50.022363 26622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:44:50.023998 26622 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:44:50.025170 26622 catalog_manager.cc:5879] T 00000000000000000000000000000000 P de3c5aeea56b4183bafc5c6d8dfc9053: Loaded TSK: 0
I20260808 11:44:50.026136 26622 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 11:44:50.284752 26589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:50.285054 26589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:50.285212 26589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:50.292484 26589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:50.292635 26589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:50.292807 26589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:44:50.302335 26589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.65.keytab
--principal=oryx/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:46759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=32865
--tserver_master_addrs=127.13.158.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:50.303620 26589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:50.304926 26589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:50.319164 26630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:50.319132 26632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:50.319835 26636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:50.320297 26589 server_base.cc:1034] running on GCE node
I20260808 11:44:50.321017 26589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:50.322126 26589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:50.323428 26589 hybrid_clock.cc:648] HybridClock initialized: now 1786189490323329 us; error 80 us; skew 500 ppm
I20260808 11:44:50.323779 26589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189490, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:50.327646 26589 init.cc:377] Logged in from keytab as oryx/127.13.158.65@KRBTEST.COM (short username oryx)
I20260808 11:44:50.329330 26589 webserver.cc:460] Webserver started at http://127.13.158.65:32865/ using document root <none> and password file <none>
I20260808 11:44:50.330291 26589 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:50.330439 26589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:50.337064 26589 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:44:50.340056 26644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:50.341369 26589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:44:50.341547 26589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "fd872afc0afd4daf893c293b7a7799cd"
format_stamp: "Formatted at 2026-08-08 11:44:47 on dist-test-slave-39l8"
server_key: "fd479db6b97d74aaa8816592185ff35d"
server_key_iv: "82346660a361b6aa063957bf4abc2403"
server_key_version: "encryptionkey@0"
I20260808 11:44:50.342464 26589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:50.365960 26589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:50.370056 26589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:50.370347 26589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:50.371368 26589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:50.372766 26589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:50.372900 26589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:50.373034 26589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:50.373131 26589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:50.406287 26589 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:46759
I20260808 11:44:50.406404 26818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:46759 every 8 connection(s)
I20260808 11:44:50.407768 26589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:44:50.409974 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 26589
I20260808 11:44:50.410791 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:37091
--local_ip_for_outbound_sockets=127.13.158.66
--tserver_master_addrs=127.13.158.126:43027
--webserver_port=41385
--webserver_interface=127.13.158.66
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.158.66
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:50.431226 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:50.411101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:41787)
0808 11:44:50.413181 (+ 2080us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:50.413228 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:50.415663 (+ 2435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:50.417320 (+ 1657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:50.417331 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:50.418090 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:50.418815 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.418835 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.421272 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.421280 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.423338 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.423357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.424678 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.424691 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.425059 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.425798 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:50.425837 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:50.429975 (+ 4138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1843,"thread_start_us":1668,"threads_started":1}
W20260808 11:44:50.431286 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:50.410909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:41787 (local address 127.13.158.126:43027)
0808 11:44:50.411868 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:50.411891 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:50.413361 (+ 1470us) server_negotiation.cc:407] Connection header received
0808 11:44:50.416306 (+ 2945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:50.416328 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:50.416756 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:50.417151 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:50.419021 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.421094 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.423485 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.424404 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.424543 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.430324 (+ 5781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:41787: BlockingRecv error: recv got EOF from 127.13.158.65:41787 (error 108)
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":291,"threads_started":1}
W20260808 11:44:50.432224 26821 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:43027 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.158.126:43027: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:50.442097 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:50.432970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:54609)
0808 11:44:50.433223 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:50.433248 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:50.433366 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:50.433672 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:50.433679 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:50.434161 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:50.434378 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.434394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.435438 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.435447 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.437109 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.437127 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.437806 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.437815 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.438039 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.438682 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:50.438710 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:50.441910 (+ 3200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 11:44:50.442373 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:50.433224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:54609 (local address 127.13.158.126:43027)
0808 11:44:50.433387 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:50.433391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:50.433408 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:44:50.433474 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:50.433477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:50.433528 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:50.433604 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:50.434537 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.435290 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.437263 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.437630 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.437744 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.442152 (+ 4408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:54609: BlockingRecv error: recv got EOF from 127.13.158.65:54609 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:50.450453 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:50.442881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:39797)
0808 11:44:50.443087 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:50.443106 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:50.443207 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:50.443498 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:50.443501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:50.443764 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:50.443894 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.443901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.445277 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.445281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.446218 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.446229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.447188 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.447192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.447307 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.447848 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:50.447865 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:50.450272 (+ 2407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260808 11:44:50.450766 26821 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:44:50.451045 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:50.443041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:39797 (local address 127.13.158.126:43027)
0808 11:44:50.443214 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:50.443220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:50.443262 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:44:50.443306 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:50.443310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:50.443384 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:50.443470 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:50.444020 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.445146 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.446354 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.447050 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.447175 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.450471 (+ 3296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:39797: BlockingRecv error: recv got EOF from 127.13.158.65:39797 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 11:44:50.744400 26827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:50.744678 26827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:50.744774 26827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:50.751828 26827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:50.751928 26827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:50.752071 26827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:44:50.761118 26827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.66.keytab
--principal=oryx/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:37091
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=41385
--tserver_master_addrs=127.13.158.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:50.762293 26827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:50.763512 26827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:50.776098 26833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:50.776103 26837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:50.776225 26834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:50.776939 26827 server_base.cc:1034] running on GCE node
I20260808 11:44:50.777505 26827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:50.778442 26827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:50.779645 26827 hybrid_clock.cc:648] HybridClock initialized: now 1786189490779610 us; error 28 us; skew 500 ppm
I20260808 11:44:50.779943 26827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189490, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:50.783041 26827 init.cc:377] Logged in from keytab as oryx/127.13.158.66@KRBTEST.COM (short username oryx)
I20260808 11:44:50.784489 26827 webserver.cc:460] Webserver started at http://127.13.158.66:41385/ using document root <none> and password file <none>
I20260808 11:44:50.785187 26827 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:50.785342 26827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:50.790544 26827 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:44:50.793285 26846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:50.794687 26827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:44:50.794914 26827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "ccb0de9f017b40b58ca626bc5db2ed31"
format_stamp: "Formatted at 2026-08-08 11:44:48 on dist-test-slave-39l8"
server_key: "8ada4531892f4bc4f14f865fce22b75f"
server_key_iv: "089e31aeb16ecf6c0a007ba2b4d20a97"
server_key_version: "encryptionkey@0"
I20260808 11:44:50.795854 26827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:50.810288 26827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:50.814139 26827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:50.814425 26827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:50.815498 26827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:50.816798 26827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:50.816933 26827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:50.817072 26827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:50.817164 26827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:50.852614 26827 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:37091
I20260808 11:44:50.852672 26997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:37091 every 8 connection(s)
I20260808 11:44:50.854055 26827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:44:50.860435 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 26827
I20260808 11:44:50.861058 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:38459
--local_ip_for_outbound_sockets=127.13.158.67
--tserver_master_addrs=127.13.158.126:43027
--webserver_port=39865
--webserver_interface=127.13.158.67
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.158.67
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:44:50.868957 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:50.856479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:46421 (local address 127.13.158.126:43027)
0808 11:44:50.856663 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:50.856667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:50.857793 (+ 1126us) server_negotiation.cc:407] Connection header received
0808 11:44:50.859643 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:50.859648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:50.859735 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:50.859841 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:50.861380 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.862039 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.863469 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.863909 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.864043 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.868695 (+ 4652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:46421: BlockingRecv error: recv got EOF from 127.13.158.66:46421 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 11:44:50.869725 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:50.857007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:46421)
0808 11:44:50.857630 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:50.857672 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:50.859401 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:50.860050 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:50.860057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:50.860523 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:50.861101 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.861116 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.862195 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.862199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.863330 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.863342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.864141 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.864149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.864376 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.865064 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:50.865085 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:50.868489 (+ 3404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":250,"threads_started":1}
W20260808 11:44:50.870309 27000 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:43027 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.158.126:43027: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:44:50.880057 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:50.871069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:51947 (local address 127.13.158.126:43027)
0808 11:44:50.871252 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:50.871259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:50.871280 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:44:50.871334 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:50.871338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:50.871391 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:50.871470 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:50.872322 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.872981 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.874907 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.875299 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.875396 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.879872 (+ 4476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:51947: BlockingRecv error: recv got EOF from 127.13.158.66:51947 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 11:44:50.880074 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:50.870902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:51947)
0808 11:44:50.871121 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:50.871140 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:50.871235 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:50.871519 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:50.871526 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:50.871957 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:50.872184 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.872197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.873119 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.873127 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.874774 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.874791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.875469 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.875477 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.875689 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.876271 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:50.876296 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:50.879734 (+ 3438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:50 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:50.889092 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:50.880802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:59349)
0808 11:44:50.881046 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:50.881070 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:50.881155 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:50.881466 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:50.881469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:50.881721 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:50.881856 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.881863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.883318 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.883322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.884206 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:50.884216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.885310 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.885314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:50.885519 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.886118 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:50.886141 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:50.888904 (+ 2763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141}
W20260808 11:44:50.889441 27000 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:44:50.889444 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:50.880957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:59349 (local address 127.13.158.126:43027)
0808 11:44:50.881163 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:50.881171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:50.881192 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:44:50.881244 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:50.881252 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:50.881330 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:50.881431 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:50.881997 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.883187 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.884349 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:50.885176 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:50.885322 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:50.889183 (+ 3861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:59349: BlockingRecv error: recv got EOF from 127.13.158.66:59349 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 11:44:51.200685 27003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:44:51.200922 27003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:44:51.201005 27003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:44:51.207937 27003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:44:51.208034 27003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:44:51.208143 27003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:44:51.217286 27003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/krb5kdc/oryx_127.13.158.67.keytab
--principal=oryx/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:38459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=39865
--tserver_master_addrs=127.13.158.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:44:51.218538 27003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:44:51.219898 27003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:44:51.232429 27013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:51.232522 27008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:44:51.232493 27009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:44:51.233644 27003 server_base.cc:1034] running on GCE node
I20260808 11:44:51.234197 27003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:44:51.235098 27003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:44:51.236321 27003 hybrid_clock.cc:648] HybridClock initialized: now 1786189491236280 us; error 51 us; skew 500 ppm
I20260808 11:44:51.236629 27003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189491, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:44:51.239675 27003 init.cc:377] Logged in from keytab as oryx/127.13.158.67@KRBTEST.COM (short username oryx)
I20260808 11:44:51.241128 27003 webserver.cc:460] Webserver started at http://127.13.158.67:39865/ using document root <none> and password file <none>
I20260808 11:44:51.241787 27003 fs_manager.cc:359] Metadata directory not provided
I20260808 11:44:51.241945 27003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:44:51.247424 27003 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 11:44:51.250015 27023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:51.251431 27003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:44:51.251613 27003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "7f18b6b67cc34e0fa3cf2f424427c41a"
format_stamp: "Formatted at 2026-08-08 11:44:48 on dist-test-slave-39l8"
server_key: "c3b3dee21f58ef0424a1d1262839fec8"
server_key_iv: "1e82b724633b06dc4f6cafc3380ffeb8"
server_key_version: "encryptionkey@0"
I20260808 11:44:51.252620 27003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:44:51.278388 27003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:44:51.282253 27003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:44:51.282559 27003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:44:51.283772 27003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:44:51.285202 27003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:44:51.285326 27003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:51.285444 27003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:44:51.285530 27003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:44:51.321580 27003 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:38459
I20260808 11:44:51.321667 27181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:38459 every 8 connection(s)
I20260808 11:44:51.323060 27003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:44:51.329710 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 27003
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:44:51.340220 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:51.325546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:45851 (local address 127.13.158.126:43027)
0808 11:44:51.325756 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:51.325761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:51.326956 (+ 1195us) server_negotiation.cc:407] Connection header received
0808 11:44:51.328825 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:51.328830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:51.328924 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:51.329039 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:51.330578 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.331720 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.334120 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.334795 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.334965 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.339975 (+ 5010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:45851: BlockingRecv error: recv got EOF from 127.13.158.67:45851 (error 108)
Metrics: {"mutex_wait_us":12,"server-negotiator.queue_time_us":68}
I20260808 11:44:51.341069 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:51.326038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:45851)
0808 11:44:51.326792 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:51.326844 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:51.328564 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:51.329277 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:51.329285 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:51.329796 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:51.330379 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.330401 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.331883 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.331891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.333987 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.333999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.335044 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.335053 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.335333 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.335968 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:51.335994 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:51.339820 (+ 3826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":270,"threads_started":1}
W20260808 11:44:51.341754 27183 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:43027 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.158.126:43027: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
I20260808 11:44:51.346673 27188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:51.330685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.13.158.126:43027)
0808 11:44:51.331382 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:51.331389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:51.331409 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:44:51.331459 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:51.331464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:51.331524 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:51.331633 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:51.333740 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.334857 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.336548 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.336907 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.336989 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.339617 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:44:51.339629 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:44:51.339646 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:44:51.339690 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:44:51.343294 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:51.344013 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:51.344026 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:51.344035 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:51.344278 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:44:51.344619 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:44:51.344626 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:44:51.344633 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:44:51.345077 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:44:51.345324 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 11:44:51.345849 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 11:44:51.346238 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":586,"thread_start_us":439,"threads_started":1}
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:51.351222 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:51.342320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:46217)
0808 11:44:51.342482 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:51.342499 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:51.342623 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:51.342953 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:51.342957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:51.343267 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:51.343468 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.343481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.344690 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.344694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.345800 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.345816 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.346822 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.346827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.346991 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.347577 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:51.347598 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:51.350950 (+ 3352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 11:44:51.351400 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:51.342402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:46217 (local address 127.13.158.126:43027)
0808 11:44:51.342541 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:51.342547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:51.342562 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:44:51.342762 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:51.342766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:51.342831 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:51.342936 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:51.343609 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.344540 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.345937 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.346588 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.346969 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.351166 (+ 4197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:46217: BlockingRecv error: recv got EOF from 127.13.158.67:46217 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:51.359033 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:51.351863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:51989)
0808 11:44:51.352018 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:51.352033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:51.352122 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:51.352413 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:51.352416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:51.352660 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:51.352796 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.352803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.353691 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.353694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.354562 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.354572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.355254 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.355259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.355385 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.355877 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:51.355900 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:51.358814 (+ 2914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 11:44:51.359186 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:51.351951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:51989 (local address 127.13.158.126:43027)
0808 11:44:51.352202 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:51.352206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:51.352221 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:44:51.352257 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:51.352260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:51.352305 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:51.352379 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:51.352912 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.353577 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.354679 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.355136 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.355234 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.358979 (+ 3745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:51989: BlockingRecv error: recv got EOF from 127.13.158.67:51989 (error 108)
Metrics: {"server-negotiator.queue_time_us":154}
W20260808 11:44:51.359304 27183 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:51.460433 27189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:51.451569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:55235)
0808 11:44:51.452122 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:51.452138 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:51.452300 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:51.452660 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:51.452665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:51.453169 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:51.453374 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.453384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.454423 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.454430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.455413 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.455425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.456155 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.456162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.456343 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.457022 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:51.457040 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:51.460202 (+ 3162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":295,"threads_started":1}
W20260808 11:44:51.460610 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:51.451656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:55235 (local address 127.13.158.126:43027)
0808 11:44:51.451875 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:51.451883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:51.452235 (+ 352us) server_negotiation.cc:407] Connection header received
0808 11:44:51.452428 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:51.452434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:51.452502 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:51.452607 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:51.453525 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.454276 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.455558 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.455966 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.456097 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.460423 (+ 4326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:55235: BlockingRecv error: recv got EOF from 127.13.158.65:55235 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:51 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:51.899056 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:51.890178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:58309)
0808 11:44:51.890699 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:51.890714 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:51.890890 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:51.891244 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:51.891250 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:51.891726 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:51.891911 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.891920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.892952 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.892958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.894126 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:51.894145 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.895050 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.895057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:51.895236 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.895855 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:51.895874 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:51.898675 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":271,"threads_started":1}
W20260808 11:44:51.899243 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:51.890270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:58309 (local address 127.13.158.126:43027)
0808 11:44:51.890503 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:51.890511 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:51.890819 (+ 308us) server_negotiation.cc:407] Connection header received
0808 11:44:51.891011 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:51.891016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:51.891087 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:51.891199 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:51.892084 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.892804 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.894275 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:51.894857 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:51.895005 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:51.898951 (+ 3946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:58309: BlockingRecv error: recv got EOF from 127.13.158.66:58309 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 08 11:44:52 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:52 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:52.372462 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:52.359914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:60781)
0808 11:44:52.360458 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:52.360481 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:52.360633 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:52.361028 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:52.361032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:52.361500 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:52.361706 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:52.361715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.362793 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.362799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:52.363718 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:52.363729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.364473 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.364480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:52.364664 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:52.369407 (+ 4743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:52.369425 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:52.372244 (+ 2819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":296,"threads_started":1}
W20260808 11:44:52.372673 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:52.360050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:60781 (local address 127.13.158.126:43027)
0808 11:44:52.360288 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:52.360296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:52.360569 (+ 273us) server_negotiation.cc:407] Connection header received
0808 11:44:52.360763 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:52.360768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:52.360876 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:52.360978 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:52.361850 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.362609 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.363855 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.364290 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.364434 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:52.372435 (+ 8001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:60781: BlockingRecv error: recv got EOF from 127.13.158.67:60781 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 11:44:52 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:52 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:52.469168 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:52.461364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:50001)
0808 11:44:52.461839 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:52.461855 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:52.461963 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:52.462266 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:52.462269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:52.462651 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:52.462844 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:52.462853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.463844 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.463847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:52.464743 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:52.464754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.465426 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.465430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:52.465539 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:52.466187 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:52.466204 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:52.468929 (+ 2725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":272,"threads_started":1}
W20260808 11:44:52.469379 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:52.461395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:50001 (local address 127.13.158.126:43027)
0808 11:44:52.461635 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:52.461647 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:52.461936 (+ 289us) server_negotiation.cc:407] Connection header received
0808 11:44:52.462059 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:52.462062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:52.462132 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:52.462214 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:52.462960 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.463719 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.464865 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.465277 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.465366 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:52.469124 (+ 3758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:50001: BlockingRecv error: recv got EOF from 127.13.158.65:50001 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 11:44:52 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:52 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:52.908990 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:52.900263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:45637)
0808 11:44:52.900837 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:52.900853 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:52.901020 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:52.901417 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:52.901422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:52.901913 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:52.902141 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:52.902151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.903525 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.903530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:52.904451 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:52.904463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.905187 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.905193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:52.905354 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:52.905919 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:52.905936 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:52.908716 (+ 2780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":316,"threads_started":1}
W20260808 11:44:52.909171 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:52.900335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:45637 (local address 127.13.158.126:43027)
0808 11:44:52.900584 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:52.900594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:52.900966 (+ 372us) server_negotiation.cc:407] Connection header received
0808 11:44:52.901148 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:52.901153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:52.901234 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:52.901339 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:52.902276 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.903385 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.904601 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:52.905019 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:52.905173 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:52.908938 (+ 3765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:45637: BlockingRecv error: recv got EOF from 127.13.158.66:45637 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 11:44:53 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:53 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:53.381798 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:53.373323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:50167)
0808 11:44:53.373944 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:53.373961 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:53.374128 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:53.374501 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:53.374506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:53.374984 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:53.375161 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:53.375170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.376218 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.376224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:53.377082 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:53.377093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.377805 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.377811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:53.377984 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:53.378655 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:53.378673 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:53.381560 (+ 2887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":350,"threads_started":1}
W20260808 11:44:53.382009 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:53.373534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:50167 (local address 127.13.158.126:43027)
0808 11:44:53.373795 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:53.373802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:53.374099 (+ 297us) server_negotiation.cc:407] Connection header received
0808 11:44:53.374275 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:53.374280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:53.374361 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:53.374467 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:53.375315 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.376075 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.377223 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.377656 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.377793 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:53.381755 (+ 3962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:50167: BlockingRecv error: recv got EOF from 127.13.158.67:50167 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 08 11:44:53 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:53 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:53.478291 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:53.470164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:53905)
0808 11:44:53.470766 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:53.470790 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:53.470905 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:53.471193 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:53.471196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:53.471545 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:53.471729 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:53.471738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.472691 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.472695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:53.473573 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:53.473584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.474254 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.474258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:53.474385 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:53.475128 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:53.475147 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:53.478032 (+ 2885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":309,"threads_started":1}
W20260808 11:44:53.478498 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:53.470232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:53905 (local address 127.13.158.126:43027)
0808 11:44:53.470406 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:53.470410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:53.470862 (+ 452us) server_negotiation.cc:407] Connection header received
0808 11:44:53.471005 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:53.471008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:53.471069 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:53.471151 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:53.471858 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.472564 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.473711 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.474090 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.474190 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:53.478250 (+ 4060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:53905: BlockingRecv error: recv got EOF from 127.13.158.65:53905 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 11:44:53 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:53 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:53.918078 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:53.909959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:46551)
0808 11:44:53.910501 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:53.910517 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:53.910629 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:53.910968 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:53.910972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:53.911320 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:53.911500 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:53.911508 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.912560 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.912563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:53.913445 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:53.913455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.914102 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.914110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:53.914264 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:53.914963 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:53.914981 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:53.917772 (+ 2791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":318,"threads_started":1}
W20260808 11:44:53.918155 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:53.910069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:46551 (local address 127.13.158.126:43027)
0808 11:44:53.910242 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:53.910247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:53.910588 (+ 341us) server_negotiation.cc:407] Connection header received
0808 11:44:53.910775 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:53.910779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:53.910843 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:53.910922 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:53.911672 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.912435 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.913560 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:53.913930 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:53.914047 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:53.917948 (+ 3901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:46551: BlockingRecv error: recv got EOF from 127.13.158.66:46551 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 11:44:54 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:54 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:54.391417 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:54.382923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:48687)
0808 11:44:54.383468 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:54.383484 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:54.383632 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:54.383996 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:54.384000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:54.384534 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:54.384748 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:54.384758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.386004 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.386015 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:54.387001 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:54.387013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.387718 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.387724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:54.387894 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:54.388472 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:54.388489 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:54.391176 (+ 2687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":324,"threads_started":1}
W20260808 11:44:54.391557 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:54.382946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:48687 (local address 127.13.158.126:43027)
0808 11:44:54.383161 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:54.383169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:54.383566 (+ 397us) server_negotiation.cc:407] Connection header received
0808 11:44:54.383776 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:54.383781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:54.383856 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:54.383958 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:54.384908 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.385863 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.387149 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.387558 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.387695 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:54.391344 (+ 3649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:48687: BlockingRecv error: recv got EOF from 127.13.158.67:48687 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 11:44:54 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:54 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:54.487423 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:54.479298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:35533)
0808 11:44:54.479751 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:54.479767 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:54.479872 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:54.480168 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:54.480171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:54.480541 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:54.480681 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:54.480739 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.481745 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.481749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:54.482670 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:54.482680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.483472 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.483476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:54.483603 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:54.484189 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:54.484206 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:54.487191 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":246,"threads_started":1}
W20260808 11:44:54.487682 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:54.479336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:35533 (local address 127.13.158.126:43027)
0808 11:44:54.479572 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:54.479578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:54.479847 (+ 269us) server_negotiation.cc:407] Connection header received
0808 11:44:54.479981 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:54.479985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:54.480046 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:54.480126 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:54.480882 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.481626 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.482867 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.483348 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.483441 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:54.487399 (+ 3958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:35533: BlockingRecv error: recv got EOF from 127.13.158.65:35533 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 11:44:54 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:54 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:54.927430 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:54.919270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:38669)
0808 11:44:54.919735 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:54.919751 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:54.919864 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:54.920151 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:54.920154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:54.920528 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:54.920676 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:54.920684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.921837 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.921841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:54.922889 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:54.922901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.923722 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.923727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:54.923910 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:54.924468 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:54.924485 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:54.927214 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":254,"threads_started":1}
W20260808 11:44:54.927604 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:54.919300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:38669 (local address 127.13.158.126:43027)
0808 11:44:54.919530 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:54.919539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:54.919832 (+ 293us) server_negotiation.cc:407] Connection header received
0808 11:44:54.919962 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:54.919966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:54.920029 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:54.920133 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:54.920830 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.921725 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.923086 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:54.923555 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:54.923698 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:54.927392 (+ 3694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:38669: BlockingRecv error: recv got EOF from 127.13.158.66:38669 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 11:44:55 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:55 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:55.401090 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:55.392571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:57371)
0808 11:44:55.393064 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:55.393080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:55.393236 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:55.393656 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:55.393660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:55.394099 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:55.394253 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:55.394261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.395466 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.395473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:55.396499 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:55.396510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.397216 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.397222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:55.397405 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:55.397963 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:55.397980 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:55.400869 (+ 2889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":276,"threads_started":1}
W20260808 11:44:55.401302 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:55.392600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:57371 (local address 127.13.158.126:43027)
0808 11:44:55.392777 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:55.392785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:55.393167 (+ 382us) server_negotiation.cc:407] Connection header received
0808 11:44:55.393410 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:55.393416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:55.393519 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:55.393627 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:55.394400 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.395325 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.396646 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.397070 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.397212 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:55.401068 (+ 3856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:57371: BlockingRecv error: recv got EOF from 127.13.158.67:57371 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 11:44:55 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:55 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:55.496424 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:55.488338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:58947)
0808 11:44:55.488842 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:55.488858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:55.488967 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:55.489302 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:55.489305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:55.489697 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:55.489876 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:55.489884 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.490921 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.490925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:55.491832 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:55.491844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.492497 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.492501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:55.492623 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:55.493191 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:55.493208 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:55.496059 (+ 2851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":240,"threads_started":1}
W20260808 11:44:55.496493 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:55.488444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:58947 (local address 127.13.158.126:43027)
0808 11:44:55.488724 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:55.488730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:55.488941 (+ 211us) server_negotiation.cc:407] Connection header received
0808 11:44:55.489108 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:55.489112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:55.489182 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:55.489262 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:55.490013 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.490796 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.491954 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.492375 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.492466 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:55.496273 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:58947: BlockingRecv error: recv got EOF from 127.13.158.65:58947 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 08 11:44:55 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:55 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:55.937544 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:55.928439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:54921)
0808 11:44:55.929003 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:55.929023 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:55.929194 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:55.929546 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:55.929551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:55.929982 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:55.930144 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:55.930153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.931247 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.931256 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:55.932208 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:55.932221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.932972 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.932977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:55.933167 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:55.933815 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:55.933832 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:55.937233 (+ 3401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":256,"threads_started":1}
W20260808 11:44:55.937681 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:55.928679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:54921 (local address 127.13.158.126:43027)
0808 11:44:55.928920 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:55.928928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:55.929123 (+ 195us) server_negotiation.cc:407] Connection header received
0808 11:44:55.929318 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:55.929325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:55.929392 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:55.929496 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:55.930299 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.931107 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.932367 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:55.932822 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:55.932962 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:55.937428 (+ 4466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:54921: BlockingRecv error: recv got EOF from 127.13.158.66:54921 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 11:44:56 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:56 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:56.410174 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:56.402022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:42551)
0808 11:44:56.402557 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:56.402573 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:56.402763 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:56.403106 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:56.403111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:56.403533 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:56.403691 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:56.403700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.404711 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.404719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:56.405660 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:56.405671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.406453 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.406458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:56.406596 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:56.407275 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:56.407293 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:56.409916 (+ 2623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":280,"threads_started":1}
W20260808 11:44:56.410344 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:56.402079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:42551 (local address 127.13.158.126:43027)
0808 11:44:56.402287 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:56.402296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:56.402662 (+ 366us) server_negotiation.cc:407] Connection header received
0808 11:44:56.402892 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:56.402897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:56.402968 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:56.403074 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:56.403839 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.404569 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.405807 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.406271 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.406396 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:56.410095 (+ 3699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:42551: BlockingRecv error: recv got EOF from 127.13.158.67:42551 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 11:44:56 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:56 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:56.505461 27210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:56.497445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:39417)
0808 11:44:56.498019 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:56.498035 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:56.498141 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:56.498448 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:56.498452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:56.498965 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:56.499116 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:56.499126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.500301 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.500304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:56.501184 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:56.501195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.501836 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.501840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:56.501953 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:56.502502 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:56.502519 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:56.505239 (+ 2720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":328,"threads_started":1}
W20260808 11:44:56.505650 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:56.497484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:39417 (local address 127.13.158.126:43027)
0808 11:44:56.497659 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:56.497663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:56.498102 (+ 439us) server_negotiation.cc:407] Connection header received
0808 11:44:56.498236 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:56.498240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:56.498319 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:56.498431 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:56.499253 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.500182 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.501329 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.501709 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.501796 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:56.505406 (+ 3610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:39417: BlockingRecv error: recv got EOF from 127.13.158.65:39417 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 11:44:56 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:56 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:56.946766 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:56.938457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:48729)
0808 11:44:56.939027 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:56.939042 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:56.939196 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:56.939532 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:56.939536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:56.939937 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:56.940096 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:56.940104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.941116 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.941121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:56.941982 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:56.941993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.942690 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.942695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:56.942888 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:56.943432 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:56.943448 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:56.946383 (+ 2935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":338,"threads_started":1}
W20260808 11:44:56.946964 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:56.938530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:48729 (local address 127.13.158.126:43027)
0808 11:44:56.938819 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:56.938827 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:56.939124 (+ 297us) server_negotiation.cc:407] Connection header received
0808 11:44:56.939325 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:56.939336 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:56.939395 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:56.939507 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:56.940242 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.940978 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.942125 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:56.942529 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:56.942663 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:56.946662 (+ 3999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:48729: BlockingRecv error: recv got EOF from 127.13.158.66:48729 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 11:44:57 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:57 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:57.424556 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:57.411305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:35331)
0808 11:44:57.412139 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:57.412159 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:57.412543 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:57.412911 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:57.412915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:57.413310 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:57.413469 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:57.413477 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.414551 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.414555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:57.415463 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:57.415474 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.416167 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.416171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:57.416289 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:57.421279 (+ 4990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:57.421297 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:57.424188 (+ 2891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":534,"threads_started":1}
W20260808 11:44:57.424644 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:57.411405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:35331 (local address 127.13.158.126:43027)
0808 11:44:57.411748 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:57.411757 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:57.412255 (+ 498us) server_negotiation.cc:407] Connection header received
0808 11:44:57.412701 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:57.412717 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:57.412787 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:57.412865 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:57.413592 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.414429 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.415588 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.416024 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.416116 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:57.424417 (+ 8301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:35331: BlockingRecv error: recv got EOF from 127.13.158.67:35331 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
Aug 08 11:44:57 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:57 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:57.514674 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:57.506556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:52511)
0808 11:44:57.507268 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:57.507285 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:57.507428 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:57.507842 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:57.507847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:57.508259 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:57.508425 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:57.508434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.509403 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.509406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:57.510294 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:57.510304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.510983 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.510987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:57.511099 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:57.511624 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:57.511640 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:57.514416 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":308,"threads_started":1}
W20260808 11:44:57.514854 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:57.506650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:52511 (local address 127.13.158.126:43027)
0808 11:44:57.506897 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:57.506904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:57.507368 (+ 464us) server_negotiation.cc:407] Connection header received
0808 11:44:57.507555 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:57.507560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:57.507660 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:57.507788 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:57.508551 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.509276 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.510404 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.510856 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.510947 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:57.514595 (+ 3648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:52511: BlockingRecv error: recv got EOF from 127.13.158.65:52511 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 11:44:57 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:57 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:57.956720 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:57.947949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:60171)
0808 11:44:57.948488 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:57.948504 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:57.948668 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:57.949017 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:57.949021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:57.949476 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:57.949654 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:57.949663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.951057 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.951064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:57.952035 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:57.952065 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.952772 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.952778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:57.952950 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:57.953475 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:57.953492 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:57.956450 (+ 2958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":231,"threads_started":1}
W20260808 11:44:57.956877 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:57.947992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:60171 (local address 127.13.158.126:43027)
0808 11:44:57.948203 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:57.948209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:57.948597 (+ 388us) server_negotiation.cc:407] Connection header received
0808 11:44:57.948815 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:57.948819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:57.948880 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:57.948987 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:57.949808 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.950925 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.952200 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:57.952611 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:57.952749 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:57.956648 (+ 3899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:60171: BlockingRecv error: recv got EOF from 127.13.158.66:60171 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 11:44:58 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:58 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:58.433874 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:58.425644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:49765)
0808 11:44:58.426216 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:58.426236 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:58.426413 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:58.426796 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:58.426800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:58.427400 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:58.427612 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:58.427641 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.428542 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.428545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:58.429414 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:58.429424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.430111 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.430117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:58.430269 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:58.430850 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:58.430867 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:58.433625 (+ 2758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":405,"threads_started":1}
W20260808 11:44:58.434027 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:58.425921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:49765 (local address 127.13.158.126:43027)
0808 11:44:58.426119 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:58.426126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:58.426313 (+ 187us) server_negotiation.cc:407] Connection header received
0808 11:44:58.426559 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:58.426563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:58.426624 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:58.426759 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:58.427770 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.428429 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.429531 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.429965 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.430051 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:58.433819 (+ 3768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:49765: BlockingRecv error: recv got EOF from 127.13.158.67:49765 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 11:44:58 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:58 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:58.523840 27219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:58.515604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:49009)
0808 11:44:58.516228 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:58.516244 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:58.516352 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:58.516636 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:58.516639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:58.517056 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:58.517233 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:58.517242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.518180 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.518184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:58.519113 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:58.519124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.519823 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.519827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:58.519934 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:58.520582 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:58.520600 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:58.523603 (+ 3003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":324,"threads_started":1}
W20260808 11:44:58.524065 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:58.515782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:49009 (local address 127.13.158.126:43027)
0808 11:44:58.515993 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:58.515997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:58.516314 (+ 317us) server_negotiation.cc:407] Connection header received
0808 11:44:58.516446 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:58.516449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:58.516509 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:58.516589 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:58.517358 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.518057 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.519267 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.519667 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.519759 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:58.523783 (+ 4024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:49009: BlockingRecv error: recv got EOF from 127.13.158.65:49009 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 11:44:58 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:58 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:58.966466 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:58.957717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:58487)
0808 11:44:58.958204 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:58.958220 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:58.958378 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:58.958743 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:58.958748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:58.959153 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:58.959298 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:58.959306 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.960429 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.960434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:58.961312 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:58.961322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.962399 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.962406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:58.962546 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:58.963302 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:58.963328 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:58.966140 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":258,"threads_started":1}
W20260808 11:44:58.966567 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:58.957812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:58487 (local address 127.13.158.126:43027)
0808 11:44:58.958058 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:58.958064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:58.958315 (+ 251us) server_negotiation.cc:407] Connection header received
0808 11:44:58.958506 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:58.958513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:58.958576 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:58.958686 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:58.959436 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.960277 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.961447 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:58.962153 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:58.962312 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:58.966329 (+ 4017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:58487: BlockingRecv error: recv got EOF from 127.13.158.66:58487 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 08 11:44:59 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:59 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:59.443406 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:59.435108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:44617)
0808 11:44:59.435663 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:59.435679 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:59.435831 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:59.436188 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:59.436192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:59.436604 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:59.436751 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:59.436760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.437814 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.437820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:59.438739 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:59.438750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.439558 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.439565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:59.439729 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:59.440328 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:59.440345 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:59.443188 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":294,"threads_started":1}
W20260808 11:44:59.443616 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:59.435188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:44617 (local address 127.13.158.126:43027)
0808 11:44:59.435423 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:59.435427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:59.435769 (+ 342us) server_negotiation.cc:407] Connection header received
0808 11:44:59.435957 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:59.435962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:59.436053 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:59.436161 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:59.436895 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.437652 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.438884 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.439336 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.439532 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:59.443369 (+ 3837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:44617: BlockingRecv error: recv got EOF from 127.13.158.67:44617 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 11:44:59 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:59 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:59.532959 27224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:59.524843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:59319)
0808 11:44:59.525382 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:59.525397 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:59.525510 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:59.525803 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:59.525806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:59.526183 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:59.526375 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:59.526384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.527375 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.527378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:59.528334 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:59.528345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.528973 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.528977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:59.529094 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:59.529673 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:59.529695 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:59.532723 (+ 3028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":303,"threads_started":1}
W20260808 11:44:59.533110 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:59.524884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:59319 (local address 127.13.158.126:43027)
0808 11:44:59.525144 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:59.525153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:59.525483 (+ 330us) server_negotiation.cc:407] Connection header received
0808 11:44:59.525594 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:59.525598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:59.525668 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:59.525759 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:59.526495 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.527251 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.528462 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.528848 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.528935 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:59.532901 (+ 3966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:59319: BlockingRecv error: recv got EOF from 127.13.158.65:59319 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 11:44:59 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:44:59 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:44:59.976089 27225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:44:59.967531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:39683)
0808 11:44:59.968023 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 11:44:59.968039 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:44:59.968230 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:44:59.968672 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:44:59.968679 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:44:59.969201 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:44:59.969470 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:59.969484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.970542 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.970545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:59.971452 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:44:59.971463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.972210 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.972214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:44:59.972330 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:59.972878 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:44:59.972894 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:44:59.975753 (+ 2859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":285,"threads_started":1}
W20260808 11:44:59.976164 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:44:59.967581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:39683 (local address 127.13.158.126:43027)
0808 11:44:59.967820 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 11:44:59.967830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:44:59.968171 (+ 341us) server_negotiation.cc:407] Connection header received
0808 11:44:59.968386 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:44:59.968409 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:44:59.968494 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:44:59.968604 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:44:59.969620 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.970420 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.971573 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:44:59.972083 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:44:59.972181 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:44:59.975938 (+ 3757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:39683: BlockingRecv error: recv got EOF from 127.13.158.66:39683 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 11:45:00 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:00 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:00.453712 27226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:00.444585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:40217)
0808 11:45:00.445154 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:00.445170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:00.445296 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:00.445662 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:00.445665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:00.446050 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:00.446324 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:00.446334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.447607 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.447611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:00.449021 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:00.449037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.449765 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.449770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:00.449902 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:00.450539 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:00.450557 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:00.453478 (+ 2921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":326,"threads_started":1}
W20260808 11:45:00.453904 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:00.444655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:40217 (local address 127.13.158.126:43027)
0808 11:45:00.444856 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:00.444860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:00.445263 (+ 403us) server_negotiation.cc:407] Connection header received
0808 11:45:00.445397 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:00.445400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:00.445529 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:00.445622 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:00.446485 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.447454 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.449192 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.449607 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.449700 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:00.453667 (+ 3967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:40217: BlockingRecv error: recv got EOF from 127.13.158.67:40217 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 11:45:00 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:00 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:00.541884 27227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:00.534061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:43947)
0808 11:45:00.534507 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:00.534523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:00.534652 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:00.534999 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:00.535003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:00.535381 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:00.535524 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:00.535533 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.536504 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.536508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:00.537411 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:00.537422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.538079 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.538083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:00.538188 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:00.538793 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:00.538810 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:00.541572 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":260,"threads_started":1}
W20260808 11:45:00.542131 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:00.534117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:43947 (local address 127.13.158.126:43027)
0808 11:45:00.534362 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:00.534369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:00.534631 (+ 262us) server_negotiation.cc:407] Connection header received
0808 11:45:00.534808 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:00.534812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:00.534874 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:00.534954 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:00.535652 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.536391 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.537530 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.537957 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.538044 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:00.541841 (+ 3797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:43947: BlockingRecv error: recv got EOF from 127.13.158.65:43947 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 11:45:00 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:00 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:00.985162 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:00.977148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:42211)
0808 11:45:00.977673 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:00.977689 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:00.977859 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:00.978260 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:00.978264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:00.978793 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:00.979018 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:00.979031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.979998 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.980001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:00.980907 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:00.980918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.981607 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.981611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:00.981729 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:00.982276 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:00.982292 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:00.984923 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":332,"threads_started":1}
W20260808 11:45:00.985374 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:00.977248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:42211 (local address 127.13.158.126:43027)
0808 11:45:00.977441 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:00.977449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:00.977800 (+ 351us) server_negotiation.cc:407] Connection header received
0808 11:45:00.978023 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:00.978034 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:00.978097 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:00.978227 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:00.979181 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.979890 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.981071 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:00.981482 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:00.981572 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:00.985100 (+ 3528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:42211: BlockingRecv error: recv got EOF from 127.13.158.66:42211 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 11:45:01 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:01 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:01.462832 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:01.454881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:57641)
0808 11:45:01.455417 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:01.455434 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:01.455574 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:01.455965 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:01.455968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:01.456335 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:01.456482 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:01.456490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.457528 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.457531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:01.458450 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:01.458460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.459170 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.459175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:01.459305 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:01.459964 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:01.459982 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:01.462556 (+ 2574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":265,"threads_started":1}
W20260808 11:45:01.463020 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:01.455053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:57641 (local address 127.13.158.126:43027)
0808 11:45:01.455330 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:01.455337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:01.455517 (+ 180us) server_negotiation.cc:407] Connection header received
0808 11:45:01.455704 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:01.455711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:01.455822 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:01.455916 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:01.456617 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.457406 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.458580 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.458992 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.459140 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:01.462808 (+ 3668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:57641: BlockingRecv error: recv got EOF from 127.13.158.67:57641 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
Aug 08 11:45:01 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:01 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:01.551915 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:01.542933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:53235)
0808 11:45:01.543489 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:01.543518 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:01.543702 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:01.544114 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:01.544119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:01.544634 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:01.544837 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:01.544847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.546062 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.546068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:01.547059 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:01.547072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.547868 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.547875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:01.548098 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:01.548811 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:01.548829 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:01.551662 (+ 2833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":277,"threads_started":1}
W20260808 11:45:01.552193 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:01.543010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:53235 (local address 127.13.158.126:43027)
0808 11:45:01.543322 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:01.543329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:01.543619 (+ 290us) server_negotiation.cc:407] Connection header received
0808 11:45:01.543840 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:01.543846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:01.543953 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:01.544085 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:01.545009 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.545902 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.547216 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.547701 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.547850 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:01.551885 (+ 4035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:53235: BlockingRecv error: recv got EOF from 127.13.158.65:53235 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 08 11:45:01 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:01 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:01.994460 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:01.986047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:53247)
0808 11:45:01.986547 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:01.986562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:01.986751 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:01.987093 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:01.987098 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:01.987531 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:01.987704 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:01.987713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.988739 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.988746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:01.989609 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:01.989619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.990429 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.990434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:01.990604 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:01.991188 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:01.991204 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:01.994230 (+ 3026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":267,"threads_started":1}
W20260808 11:45:01.994748 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:01.986137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:53247 (local address 127.13.158.126:43027)
0808 11:45:01.986375 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:01.986382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:01.986658 (+ 276us) server_negotiation.cc:407] Connection header received
0808 11:45:01.986878 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:01.986883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:01.986954 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:01.987058 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:01.987881 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.988603 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.989782 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:01.990267 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:01.990398 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:01.994452 (+ 4054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:53247: BlockingRecv error: recv got EOF from 127.13.158.66:53247 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 08 11:45:02 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:02 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:02.476701 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:02.463756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:50979)
0808 11:45:02.464263 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:02.464278 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:02.464387 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:02.464664 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:02.464667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:02.465085 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:02.465271 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:02.465280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.466243 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.466247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:02.467116 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:02.467127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.467775 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.467779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:02.467908 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:02.472943 (+ 5035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:02.472962 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:02.476356 (+ 3394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":276,"threads_started":1}
W20260808 11:45:02.476779 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:02.463846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:50979 (local address 127.13.158.126:43027)
0808 11:45:02.464022 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:02.464026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:02.464346 (+ 320us) server_negotiation.cc:407] Connection header received
0808 11:45:02.464477 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:02.464481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:02.464541 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:02.464619 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:02.465399 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.466132 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.467247 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.467651 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.467739 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:02.476543 (+ 8804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:50979: BlockingRecv error: recv got EOF from 127.13.158.67:50979 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 11:45:02 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:02 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:02.561614 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:02.552808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:34473)
0808 11:45:02.553303 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:02.553319 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:02.553429 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:02.553723 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:02.553726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:02.554100 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:02.554251 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:02.554259 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.555212 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.555215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:02.556212 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:02.556230 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.557080 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.557085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:02.557289 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:02.558107 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:02.558130 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:02.561390 (+ 3260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":274,"threads_started":1}
W20260808 11:45:02.561798 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:02.552830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:34473 (local address 127.13.158.126:43027)
0808 11:45:02.553048 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:02.553059 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:02.553401 (+ 342us) server_negotiation.cc:407] Connection header received
0808 11:45:02.553510 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:02.553513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:02.553587 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:02.553665 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:02.554367 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.555094 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.556401 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.556957 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.557051 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:02.561581 (+ 4530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:34473: BlockingRecv error: recv got EOF from 127.13.158.65:34473 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 11:45:02 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:02 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:03.004843 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:02.995435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:41151)
0808 11:45:02.996039 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:02.996059 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:02.996239 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:02.996647 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:02.996653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:02.997109 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:02.997270 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:02.997279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.998506 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.998529 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:02.999824 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:02.999838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.000685 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.000692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:03.000890 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:03.001502 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:03.001531 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:03.004572 (+ 3041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":335,"threads_started":1}
W20260808 11:45:03.005012 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:02.995598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:41151 (local address 127.13.158.126:43027)
0808 11:45:02.995872 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:02.995880 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:02.996160 (+ 280us) server_negotiation.cc:407] Connection header received
0808 11:45:02.996381 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:02.996387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:02.996474 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:02.996599 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:02.997430 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:02.998352 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:02.999989 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.000512 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.000661 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:03.004783 (+ 4122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:41151: BlockingRecv error: recv got EOF from 127.13.158.66:41151 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 11:45:03 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:03 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:03.486146 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:03.477857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:50519)
0808 11:45:03.478415 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:03.478431 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:03.478546 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:03.478887 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:03.478890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:03.479276 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:03.479425 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:03.479433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.480407 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.480411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:03.481351 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:03.481362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.482001 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.482006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:03.482132 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:03.482757 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:03.482780 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:03.485798 (+ 3018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":333,"threads_started":1}
W20260808 11:45:03.486261 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:03.477900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:50519 (local address 127.13.158.126:43027)
0808 11:45:03.478089 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:03.478093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:03.478508 (+ 415us) server_negotiation.cc:407] Connection header received
0808 11:45:03.478653 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:03.478656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:03.478735 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:03.478825 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:03.479563 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.480285 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.481472 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.481859 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.481947 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:03.485999 (+ 4052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:50519: BlockingRecv error: recv got EOF from 127.13.158.67:50519 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 11:45:03 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:03 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:03.571166 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:03.562664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:35961)
0808 11:45:03.563238 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:03.563253 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:03.563364 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:03.563682 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:03.563686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:03.564048 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:03.564201 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:03.564210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.565158 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.565161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:03.566231 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:03.566245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.567235 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.567242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:03.567452 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:03.568134 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:03.568152 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:03.570905 (+ 2753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":306,"threads_started":1}
W20260808 11:45:03.571380 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:03.562954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:35961 (local address 127.13.158.126:43027)
0808 11:45:03.563171 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:03.563179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:03.563328 (+ 149us) server_negotiation.cc:407] Connection header received
0808 11:45:03.563458 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:03.563461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:03.563532 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:03.563606 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:03.564324 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.565040 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.566376 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:03.567037 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:03.567184 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:03.571128 (+ 3944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:35961: BlockingRecv error: recv got EOF from 127.13.158.65:35961 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 11:45:04 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:04 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:04.015862 27242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:04.005813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:59633)
0808 11:45:04.006432 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:04.006448 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:04.006604 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:04.006986 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:04.006991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:04.007442 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:04.007593 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:04.007601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.008616 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.008622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:04.009605 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:04.009616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.010377 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.010384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:04.010560 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:04.011275 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:04.011292 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:04.015459 (+ 4167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":338,"threads_started":1}
W20260808 11:45:04.016022 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:04.006052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:59633 (local address 127.13.158.126:43027)
0808 11:45:04.006301 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:04.006308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:04.006535 (+ 227us) server_negotiation.cc:407] Connection header received
0808 11:45:04.006769 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:04.006774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:04.006847 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:04.006954 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:04.007744 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.008476 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.009765 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.010191 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.010339 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:04.015783 (+ 5444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:59633: BlockingRecv error: recv got EOF from 127.13.158.66:59633 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 08 11:45:04 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:04 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:04.495582 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:04.487202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:37297)
0808 11:45:04.487742 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:04.487758 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:04.487869 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:04.488193 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:04.488196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:04.488593 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:04.488752 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:04.488760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.489721 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.489724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:04.490643 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:04.490654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.491372 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.491377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:04.491498 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:04.492103 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:04.492120 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:04.495228 (+ 3108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":277,"threads_started":1}
W20260808 11:45:04.495678 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:04.487280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:37297 (local address 127.13.158.126:43027)
0808 11:45:04.487448 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:04.487452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:04.487829 (+ 377us) server_negotiation.cc:407] Connection header received
0808 11:45:04.487991 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:04.487994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:04.488054 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:04.488135 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:04.488878 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.489592 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.490812 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.491217 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.491308 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:04.495435 (+ 4127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:37297: BlockingRecv error: recv got EOF from 127.13.158.67:37297 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 11:45:04 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:04 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:04.580463 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:04.572090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:39399)
0808 11:45:04.572595 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:04.572610 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:04.572725 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:04.573020 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:04.573023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:04.573401 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:04.573575 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:04.573583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.574555 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.574559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:04.575525 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:04.575537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.576225 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.576229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:04.576353 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:04.576939 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:04.576957 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:04.580090 (+ 3133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":278,"threads_started":1}
W20260808 11:45:04.580602 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:04.572233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:39399 (local address 127.13.158.126:43027)
0808 11:45:04.572464 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:04.572468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:04.572665 (+ 197us) server_negotiation.cc:407] Connection header received
0808 11:45:04.572819 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:04.572822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:04.572897 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:04.572974 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:04.573700 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.574434 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.575649 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:04.576084 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:04.576186 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:04.580338 (+ 4152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:39399: BlockingRecv error: recv got EOF from 127.13.158.65:39399 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 08 11:45:05 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:05 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:05.025110 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:05.016895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:46267)
0808 11:45:05.017390 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:05.017405 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:05.017521 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:05.017840 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:05.017844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:05.018228 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:05.018397 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:05.018406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.019738 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.019744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:05.020640 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:05.020652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.021260 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.021264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:05.021377 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:05.022026 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:05.022044 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:05.024856 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":271,"threads_started":1}
W20260808 11:45:05.025274 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:05.016991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:46267 (local address 127.13.158.126:43027)
0808 11:45:05.017163 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:05.017167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:05.017481 (+ 314us) server_negotiation.cc:407] Connection header received
0808 11:45:05.017619 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:05.017622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:05.017680 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:05.017766 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:05.018536 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.019576 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.020768 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.021130 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.021218 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:05.025065 (+ 3847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:46267: BlockingRecv error: recv got EOF from 127.13.158.66:46267 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 11:45:05 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:05 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.67@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:05.505322 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:05.496624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.67:33493)
0808 11:45:05.497232 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:05.497253 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:05.497424 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:05.497835 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:05.497840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:05.498336 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:05.498654 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:05.498671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.499852 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.499856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:05.500871 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:05.500883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.501503 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.501507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:05.501619 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:05.502337 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:05.502356 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:05.505081 (+ 2725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":343,"threads_started":1}
W20260808 11:45:05.505563 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:05.496739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:33493 (local address 127.13.158.126:43027)
0808 11:45:05.496955 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:05.496963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:05.497370 (+ 407us) server_negotiation.cc:407] Connection header received
0808 11:45:05.497547 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:05.497554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:05.497660 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:05.497783 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:05.498834 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.499725 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.500992 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.501386 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.501477 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:05.505303 (+ 3826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.67:33493: BlockingRecv error: recv got EOF from 127.13.158.67:33493 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 11:45:05 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:05 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.65@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:05.590433 27248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:05.581833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.65:48319)
0808 11:45:05.582525 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:05.582546 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:05.582654 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:05.583001 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:05.583004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:05.583428 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:05.583591 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:05.583600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.584591 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.584595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:05.585758 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:05.585772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.586496 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.586500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:05.586609 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:05.587348 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:05.587365 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:05.590171 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":390,"threads_started":1}
W20260808 11:45:05.590622 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:05.581893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:48319 (local address 127.13.158.126:43027)
0808 11:45:05.582054 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:05.582058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:05.582624 (+ 566us) server_negotiation.cc:407] Connection header received
0808 11:45:05.582790 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:05.582794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:05.582882 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:05.582959 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:05.583728 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.584431 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.585891 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:05.586361 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:05.586470 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:05.590361 (+ 3891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.65:48319: BlockingRecv error: recv got EOF from 127.13.158.65:48319 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 11:45:06 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:45:06 dist-test-slave-39l8 krb5kdc[25721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.158.66@KRBTEST.COM for oryx/127.13.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:45:06.034376 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:06.026354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43027 (local address 127.13.158.66:58149)
0808 11:45:06.026928 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:06.026944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:06.027055 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:06.027353 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:06.027357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:06.027755 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:06.027917 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:06.027926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.028836 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.028840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:06.029740 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:06.029751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.030397 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.030401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:06.030515 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:06.031117 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:06.031152 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:06.034021 (+ 2869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.158.126:43027: Server oryx/127.13.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":291,"threads_started":1}
W20260808 11:45:06.034509 26823 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:06.026453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:58149 (local address 127.13.158.126:43027)
0808 11:45:06.026682 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:06.026688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:06.027022 (+ 334us) server_negotiation.cc:407] Connection header received
0808 11:45:06.027158 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:06.027162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:06.027233 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:06.027306 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:06.028038 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.028715 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.029858 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.030281 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.030366 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:06.034250 (+ 3884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.158.66:58149: BlockingRecv error: recv got EOF from 127.13.158.66:58149 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:45:06.341032 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 26589
I20260808 11:45:06.353367 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:06.342052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44774 (local address 127.13.158.65:46759)
0808 11:45:06.342662 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:06.342669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:06.342696 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:45:06.342807 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:06.342814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:06.343034 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:06.343177 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:06.344703 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.345844 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.347227 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.347777 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.347883 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:06.350432 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:06.350441 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:06.350454 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:06.350484 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:06.352888 (+ 2404us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.158.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 11:45:06.353144 (+ 256us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44774: Request ticket server kudu/127.13.158.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":315,"threads_started":1}
W20260808 11:45:06.353497 27251 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44774: Request ticket server kudu/127.13.158.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 11:45:06.353523 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 26589: Not authorized: Client connection negotiation failed: client connection to 127.13.158.65:46759: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.158.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 11:45:06.353775 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 26589
I20260808 11:45:06.377117 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 26827
I20260808 11:45:06.389866 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:06.378148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.13.158.66:37091)
0808 11:45:06.378682 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:06.378689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:06.378773 (+ 84us) server_negotiation.cc:407] Connection header received
0808 11:45:06.378838 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:06.378844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:06.379063 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:06.379214 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:06.380383 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.381420 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.383286 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.383940 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.384100 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:06.386813 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:06.386824 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:06.386844 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:06.386874 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:06.389333 (+ 2459us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.158.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 11:45:06.389575 (+ 242us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52030: Request ticket server kudu/127.13.158.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":240,"threads_started":1}
W20260808 11:45:06.389983 27254 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52030: Request ticket server kudu/127.13.158.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 11:45:06.390012 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 26827: Not authorized: Client connection negotiation failed: client connection to 127.13.158.66:37091: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.158.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 11:45:06.390174 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 26827
I20260808 11:45:06.421025 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 27003
I20260808 11:45:06.432513 27256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:06.421936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.13.158.67:38459)
0808 11:45:06.422552 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:06.422559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:06.422585 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:45:06.422649 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:06.422655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:06.422924 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:06.423085 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:06.424321 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.425298 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.426407 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:06.427055 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:06.427158 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:06.429626 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:06.429634 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:06.429646 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:06.429682 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:06.432023 (+ 2341us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.158.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 11:45:06.432280 (+ 257us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42760: Request ticket server kudu/127.13.158.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":320,"threads_started":1}
W20260808 11:45:06.432631 27256 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42760: Request ticket server kudu/127.13.158.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 11:45:06.432691 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 27003: Not authorized: Client connection negotiation failed: client connection to 127.13.158.67:38459: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.158.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 11:45:06.432849 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 27003
I20260808 11:45:06.456743 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 26483
I20260808 11:45:06.458005 26580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:06.573428 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 26483
2026-08-08T11:45:06Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21411 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T11:45:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:45:06Z Disabled control of system clock
I20260808 11:45:06.663625 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:40351
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:40351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 11:45:06.994269 27262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:06.994555 27262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:06.994633 27262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:07.001750 27262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:45:07.001847 27262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:07.001885 27262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:07.001936 27262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:45:07.001972 27262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:45:07.011293 27262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:40351
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:40351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:07.012653 27262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:07.014501 27262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:07.025906 27268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:07.025939 27271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:07.025961 27269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:07.026798 27262 server_base.cc:1034] running on GCE node
I20260808 11:45:07.027506 27262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:07.028815 27262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:07.030040 27262 hybrid_clock.cc:648] HybridClock initialized: now 1786189507030000 us; error 48 us; skew 500 ppm
I20260808 11:45:07.030408 27262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:45:07.032756 27262 webserver.cc:460] Webserver started at http://127.13.158.126:35349/ using document root <none> and password file <none>
I20260808 11:45:07.033448 27262 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:07.033582 27262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:07.033917 27262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:07.036743 27262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "ea05108037ba4f0b92d2dbcf44e78c37"
format_stamp: "Formatted at 2026-08-08 11:45:07 on dist-test-slave-39l8"
server_key: "bf756e6db5e0d97d41e835aeb1a2872e"
server_key_iv: "ddfcb8ea5fce4a9d9a829aabd0131a31"
server_key_version: "encryptionkey@0"
I20260808 11:45:07.037503 27262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "ea05108037ba4f0b92d2dbcf44e78c37"
format_stamp: "Formatted at 2026-08-08 11:45:07 on dist-test-slave-39l8"
server_key: "bf756e6db5e0d97d41e835aeb1a2872e"
server_key_iv: "ddfcb8ea5fce4a9d9a829aabd0131a31"
server_key_version: "encryptionkey@0"
I20260808 11:45:07.042495 27262 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:45:07.045989 27280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:07.047348 27262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:45:07.047564 27262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "ea05108037ba4f0b92d2dbcf44e78c37"
format_stamp: "Formatted at 2026-08-08 11:45:07 on dist-test-slave-39l8"
server_key: "bf756e6db5e0d97d41e835aeb1a2872e"
server_key_iv: "ddfcb8ea5fce4a9d9a829aabd0131a31"
server_key_version: "encryptionkey@0"
I20260808 11:45:07.047792 27262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:07.068174 27262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:07.069028 27262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:07.069314 27262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:07.087924 27262 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:40351
I20260808 11:45:07.087958 27351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:40351 every 8 connection(s)
I20260808 11:45:07.089278 27262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:45:07.092945 27353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:45:07.095486 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 27262
I20260808 11:45:07.095690 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:45:07.096520 13945 external_mini_cluster.cc:1428] Setting key 955f44479fcaf3576bc21f849b88ad04
I20260808 11:45:07.105392 27353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37: Bootstrap starting.
I20260808 11:45:07.109071 27353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37: Neither blocks nor log segments found. Creating new log.
I20260808 11:45:07.110333 27353 log.cc:824] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37: Log is configured to *not* fsync() on all Append() calls
I20260808 11:45:07.110769 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:07.098615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.13.158.126:40351)
0808 11:45:07.099740 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:07.099759 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:07.099799 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:45:07.101408 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:07.101429 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:07.101881 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:07.102189 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:07.102932 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:07.104220 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:07.106162 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:07.106908 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:07.107048 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:07.108312 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:07.108320 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:07.108333 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:07.108830 (+ 497us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:07.108931 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:07.108954 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:07.109055 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:07.109345 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 11:45:07.109700 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":408,"threads_started":1}
I20260808 11:45:07.113643 27353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37: No bootstrap required, opened a new log
I20260808 11:45:07.120569 27353 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea05108037ba4f0b92d2dbcf44e78c37" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 40351 } }
I20260808 11:45:07.120927 27353 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:45:07.121037 27353 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea05108037ba4f0b92d2dbcf44e78c37, State: Initialized, Role: FOLLOWER
I20260808 11:45:07.121562 27353 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea05108037ba4f0b92d2dbcf44e78c37" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 40351 } }
I20260808 11:45:07.121795 27353 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:45:07.121950 27353 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:45:07.122125 27353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:45:07.124245 27353 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea05108037ba4f0b92d2dbcf44e78c37" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 40351 } }
I20260808 11:45:07.124801 27353 leader_election.cc:302] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea05108037ba4f0b92d2dbcf44e78c37; no voters:
I20260808 11:45:07.125433 27353 leader_election.cc:288] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:45:07.125581 27364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:45:07.126276 27364 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [term 1 LEADER]: Becoming Leader. State: Replica: ea05108037ba4f0b92d2dbcf44e78c37, State: Running, Role: LEADER
I20260808 11:45:07.126827 27364 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea05108037ba4f0b92d2dbcf44e78c37" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 40351 } }
I20260808 11:45:07.127497 27353 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:45:07.129451 27367 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea05108037ba4f0b92d2dbcf44e78c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea05108037ba4f0b92d2dbcf44e78c37" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 40351 } } }
I20260808 11:45:07.130034 27367 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [sys.catalog]: This master's current role is: LEADER
I20260808 11:45:07.129402 27368 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea05108037ba4f0b92d2dbcf44e78c37. Latest consensus state: current_term: 1 leader_uuid: "ea05108037ba4f0b92d2dbcf44e78c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea05108037ba4f0b92d2dbcf44e78c37" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 40351 } } }
I20260808 11:45:07.130546 27368 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37 [sys.catalog]: This master's current role is: LEADER
I20260808 11:45:07.132409 27376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:45:07.137380 27376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:45:07.146654 27376 catalog_manager.cc:1347] Generated new cluster ID: a03ba2674ee342a0897fc5127b454166
I20260808 11:45:07.146806 27376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:45:07.162576 27376 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:45:07.164387 27376 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:45:07.178306 27376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea05108037ba4f0b92d2dbcf44e78c37: Generated new TSK 0
I20260808 11:45:07.179338 27376 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:45:07.197121 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:40351
--builtin_ntp_servers=127.13.158.84:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 11:45:07.536499 27388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:07.536741 27388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:07.536885 27388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:07.544031 27388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:07.544124 27388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:07.544258 27388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:45:07.553287 27388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:40351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:07.554488 27388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:07.556267 27388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:07.569571 27393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:07.569795 27396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:07.569796 27394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:07.570613 27388 server_base.cc:1034] running on GCE node
I20260808 11:45:07.571287 27388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:07.572265 27388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:07.573516 27388 hybrid_clock.cc:648] HybridClock initialized: now 1786189507573465 us; error 57 us; skew 500 ppm
I20260808 11:45:07.573881 27388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:45:07.580124 27388 webserver.cc:460] Webserver started at http://127.13.158.65:37935/ using document root <none> and password file <none>
I20260808 11:45:07.580874 27388 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:07.581013 27388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:07.581357 27388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:07.584398 27388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "d5dd9e8cb5a04a2f892482b66f794fc1"
format_stamp: "Formatted at 2026-08-08 11:45:07 on dist-test-slave-39l8"
server_key: "fef4e14a34e1f7bfd9791bab870c8906"
server_key_iv: "39619fb804cd4e292ac54bcbe56d96f2"
server_key_version: "encryptionkey@0"
I20260808 11:45:07.585265 27388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "d5dd9e8cb5a04a2f892482b66f794fc1"
format_stamp: "Formatted at 2026-08-08 11:45:07 on dist-test-slave-39l8"
server_key: "fef4e14a34e1f7bfd9791bab870c8906"
server_key_iv: "39619fb804cd4e292ac54bcbe56d96f2"
server_key_version: "encryptionkey@0"
I20260808 11:45:07.590189 27388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:45:07.593647 27405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:07.595042 27388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:45:07.595232 27388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "d5dd9e8cb5a04a2f892482b66f794fc1"
format_stamp: "Formatted at 2026-08-08 11:45:07 on dist-test-slave-39l8"
server_key: "fef4e14a34e1f7bfd9791bab870c8906"
server_key_iv: "39619fb804cd4e292ac54bcbe56d96f2"
server_key_version: "encryptionkey@0"
I20260808 11:45:07.595454 27388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:07.646642 27388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:07.647586 27388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:07.647902 27388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:07.648967 27388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:07.650435 27388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:07.650557 27388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:07.650689 27388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:07.650800 27388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 11:45:07.685386 27388 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:43695
I20260808 11:45:07.685407 27565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:43695 every 8 connection(s)
I20260808 11:45:07.686666 27388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:45:07.695782 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 27388
I20260808 11:45:07.695999 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:45:07.696671 13945 external_mini_cluster.cc:1428] Setting key d4decb601ecbdd95f3533181ad26a32c
I20260808 11:45:07.699010 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:07.689094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:51725 (local address 127.13.158.126:40351)
0808 11:45:07.689528 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:07.689533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:07.690550 (+ 1017us) server_negotiation.cc:407] Connection header received
0808 11:45:07.692397 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:07.692404 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:07.692497 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:07.692620 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:07.693660 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:07.694333 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:07.695968 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:07.696441 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:07.696570 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:07.697999 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:07.698005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:07.698010 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:07.698030 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:07.698075 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:07.698090 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:07.698195 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:07.698540 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 11:45:07.698645 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":227,"threads_started":1}
I20260808 11:45:07.699674 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:40351
--builtin_ntp_servers=127.13.158.84:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 11:45:07.700179 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:07.689529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:40351 (local address 127.13.158.65:51725)
0808 11:45:07.690395 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:07.690439 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:07.692119 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:07.692842 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:07.692850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:07.692928 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:07.693486 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:07.693502 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:07.694503 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:07.694507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:07.695804 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:07.695818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:07.696654 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:07.696661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:07.696944 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:07.697701 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:45:07.697703 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:07.697740 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:45:07.697818 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:07.698274 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:07.698288 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:07.698298 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 11:45:07.698538 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 11:45:07.698947 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":260,"threads_started":1}
I20260808 11:45:07.701598 27566 heartbeater.cc:347] Connected to a master server at 127.13.158.126:40351
I20260808 11:45:07.701922 27566 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:07.702826 27566 heartbeater.cc:511] Master 127.13.158.126:40351 requested a full tablet report, sending...
I20260808 11:45:07.705081 27302 ts_manager.cc:194] Registered new tserver with Master: d5dd9e8cb5a04a2f892482b66f794fc1 (127.13.158.65:43695)
I20260808 11:45:07.707279 27302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.65:51725
W20260808 11:45:08.055823 27573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:08.056504 27573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:08.056746 27573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:08.070400 27573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:08.070618 27573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:08.070922 27573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:45:08.088181 27573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:40351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:08.090405 27573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:08.093652 27573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:08.120560 27581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:08.120401 27582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:08.120520 27584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:08.122583 27573 server_base.cc:1034] running on GCE node
I20260808 11:45:08.124468 27573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:08.126282 27573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:08.127794 27573 hybrid_clock.cc:648] HybridClock initialized: now 1786189508127692 us; error 120 us; skew 500 ppm
I20260808 11:45:08.128474 27573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:45:08.132752 27573 webserver.cc:460] Webserver started at http://127.13.158.66:37371/ using document root <none> and password file <none>
I20260808 11:45:08.134007 27573 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:08.134246 27573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:08.134812 27573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:08.140025 27573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "b0759bdc1f824d13b67c1e79ee90589f"
format_stamp: "Formatted at 2026-08-08 11:45:08 on dist-test-slave-39l8"
server_key: "d22cd13faa6ca0b902a848c4fefae98c"
server_key_iv: "84afd0d750334750b0f9a4cfff225188"
server_key_version: "encryptionkey@0"
I20260808 11:45:08.141378 27573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "b0759bdc1f824d13b67c1e79ee90589f"
format_stamp: "Formatted at 2026-08-08 11:45:08 on dist-test-slave-39l8"
server_key: "d22cd13faa6ca0b902a848c4fefae98c"
server_key_iv: "84afd0d750334750b0f9a4cfff225188"
server_key_version: "encryptionkey@0"
I20260808 11:45:08.152172 27573 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.010s sys 0.001s
I20260808 11:45:08.158479 27592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:08.161063 27573 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260808 11:45:08.161375 27573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "b0759bdc1f824d13b67c1e79ee90589f"
format_stamp: "Formatted at 2026-08-08 11:45:08 on dist-test-slave-39l8"
server_key: "d22cd13faa6ca0b902a848c4fefae98c"
server_key_iv: "84afd0d750334750b0f9a4cfff225188"
server_key_version: "encryptionkey@0"
I20260808 11:45:08.162127 27573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:08.193521 27573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:08.194988 27573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:08.195619 27573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:08.197563 27573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:08.200958 27573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:08.201160 27573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:08.201381 27573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:08.201606 27573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:08.254523 27573 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:45675
I20260808 11:45:08.254612 27756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:45675 every 8 connection(s)
I20260808 11:45:08.255878 27573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:45:08.261101 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 27573
I20260808 11:45:08.261269 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:45:08.261827 13945 external_mini_cluster.cc:1428] Setting key f806fb1580468a93288262eed4d0c3a6
I20260808 11:45:08.264695 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:40351
--builtin_ntp_servers=127.13.158.84:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 11:45:08.270450 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:08.258509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:45017 (local address 127.13.158.126:40351)
0808 11:45:08.259015 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:08.259022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:08.260180 (+ 1158us) server_negotiation.cc:407] Connection header received
0808 11:45:08.262528 (+ 2348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:08.262535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:08.262625 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:08.262803 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:08.263982 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.264834 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.267465 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.267879 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.267977 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:08.269640 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:08.269645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:08.269648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:08.269668 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:08.269715 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:08.269726 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:08.269795 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:08.270066 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 11:45:08.270172 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":254,"threads_started":1}
I20260808 11:45:08.271914 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:08.259141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:40351 (local address 127.13.158.66:45017)
0808 11:45:08.260003 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:08.260042 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:08.262226 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:08.263038 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:08.263047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:08.263134 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:08.263773 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:08.263815 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.265072 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.265080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:08.267296 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:08.267314 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.268110 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.268120 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:08.268444 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:08.269266 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:45:08.269271 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:08.269326 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:45:08.269447 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:08.269805 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:08.269817 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:08.269828 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 11:45:08.270046 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 11:45:08.270463 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":254,"threads_started":1}
I20260808 11:45:08.273407 27757 heartbeater.cc:347] Connected to a master server at 127.13.158.126:40351
I20260808 11:45:08.273752 27757 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:08.274817 27757 heartbeater.cc:511] Master 127.13.158.126:40351 requested a full tablet report, sending...
I20260808 11:45:08.276504 27302 ts_manager.cc:194] Registered new tserver with Master: b0759bdc1f824d13b67c1e79ee90589f (127.13.158.66:45675)
I20260808 11:45:08.277712 27302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.66:45017
W20260808 11:45:08.602239 27763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:08.602577 27763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:08.602672 27763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:08.613677 27763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:08.613831 27763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:08.614142 27763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:45:08.624910 27763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:40351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:08.626212 27763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:08.628126 27763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:08.641806 27773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:08.641773 27774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:08.641846 27777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:08.642515 27763 server_base.cc:1034] running on GCE node
I20260808 11:45:08.643168 27763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:08.644137 27763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:08.645364 27763 hybrid_clock.cc:648] HybridClock initialized: now 1786189508645317 us; error 44 us; skew 500 ppm
I20260808 11:45:08.645690 27763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:45:08.647902 27763 webserver.cc:460] Webserver started at http://127.13.158.67:45099/ using document root <none> and password file <none>
I20260808 11:45:08.648607 27763 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:08.648723 27763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:08.649073 27763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:08.652026 27763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "7856ea0beee84fe39942b2c4db12fb45"
format_stamp: "Formatted at 2026-08-08 11:45:08 on dist-test-slave-39l8"
server_key: "f8682f0539017420555e2042b00667b7"
server_key_iv: "6fded94b36d9c67dc4c571a9c1737290"
server_key_version: "encryptionkey@0"
I20260808 11:45:08.652829 27763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "7856ea0beee84fe39942b2c4db12fb45"
format_stamp: "Formatted at 2026-08-08 11:45:08 on dist-test-slave-39l8"
server_key: "f8682f0539017420555e2042b00667b7"
server_key_iv: "6fded94b36d9c67dc4c571a9c1737290"
server_key_version: "encryptionkey@0"
I20260808 11:45:08.658064 27763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260808 11:45:08.661629 27791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:08.663177 27763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:45:08.663319 27763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "7856ea0beee84fe39942b2c4db12fb45"
format_stamp: "Formatted at 2026-08-08 11:45:08 on dist-test-slave-39l8"
server_key: "f8682f0539017420555e2042b00667b7"
server_key_iv: "6fded94b36d9c67dc4c571a9c1737290"
server_key_version: "encryptionkey@0"
I20260808 11:45:08.663563 27763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:08.699826 27763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:08.700678 27763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:08.700984 27763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:08.701933 27763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:08.703429 27763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:08.703521 27763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:08.703687 27763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:08.703783 27763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:08.710997 27566 heartbeater.cc:503] Master 127.13.158.126:40351 was elected leader, sending a full tablet report...
I20260808 11:45:08.737062 27763 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:45589
I20260808 11:45:08.737175 27949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:45589 every 8 connection(s)
I20260808 11:45:08.738430 27763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:45:08.746995 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 27763
I20260808 11:45:08.747160 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:45:08.747694 13945 external_mini_cluster.cc:1428] Setting key d242052f132b5e0a7f740a689a2c4d9d
I20260808 11:45:08.751345 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:08.740910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:60287 (local address 127.13.158.126:40351)
0808 11:45:08.741089 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:08.741094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:08.742429 (+ 1335us) server_negotiation.cc:407] Connection header received
0808 11:45:08.744188 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:08.744194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:08.744289 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:08.744404 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:08.745465 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.746224 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.748455 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.748942 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.749088 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:08.750458 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:08.750465 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:08.750469 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:08.750488 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:08.750526 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:08.750544 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:08.750653 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:08.750918 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 11:45:08.751011 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:45:08.752287 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:08.741421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:40351 (local address 127.13.158.67:60287)
0808 11:45:08.742273 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:08.742306 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:08.743918 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:08.744631 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:08.744638 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:08.744721 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:08.745277 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:08.745294 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.746374 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.746380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:08.748304 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:08.748316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.749170 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.749177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:08.749439 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:08.750156 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:45:08.750159 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:08.750195 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:45:08.750274 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:08.750654 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:08.750667 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:08.750677 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 11:45:08.750904 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 11:45:08.751278 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":267,"threads_started":1}
I20260808 11:45:08.753564 27950 heartbeater.cc:347] Connected to a master server at 127.13.158.126:40351
I20260808 11:45:08.753835 27950 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:08.754504 27950 heartbeater.cc:511] Master 127.13.158.126:40351 requested a full tablet report, sending...
I20260808 11:45:08.755892 27302 ts_manager.cc:194] Registered new tserver with Master: 7856ea0beee84fe39942b2c4db12fb45 (127.13.158.67:45589)
I20260808 11:45:08.756695 27302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.67:60287
I20260808 11:45:08.763022 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 11:45:08.769389 27759 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:08.767592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.13.158.126:40351)
0808 11:45:08.767760 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:08.767764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:08.768249 (+ 485us) server_negotiation.cc:407] Connection header received
0808 11:45:08.768440 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:08.768444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:08.768532 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:08.768652 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:08.769141 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43640: BlockingRecv error: recv got EOF from 127.0.0.1:43640 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:45:08.769776 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 27388
I20260808 11:45:08.777748 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:08.771100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.13.158.65:43695)
0808 11:45:08.771633 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:08.771642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:08.771668 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:45:08.771729 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:08.771735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:08.771926 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:08.772085 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:08.772462 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.773566 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.774785 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.775347 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.775446 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:08.776443 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:08.776451 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:08.776464 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:08.777052 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:08.777097 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:08.777121 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:08.777220 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:08.777346 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 11:45:08.777458 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":265,"threads_started":1}
I20260808 11:45:08.779012 27558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:08.859385 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 27388
I20260808 11:45:08.887667 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 27573
I20260808 11:45:08.895388 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:08.888628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.13.158.66:45675)
0808 11:45:08.889192 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:08.889199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:08.889232 (+ 33us) server_negotiation.cc:407] Connection header received
0808 11:45:08.889292 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:08.889298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:08.889513 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:08.889638 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:08.889980 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.891042 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.892274 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:08.892805 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:08.892904 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:08.893862 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:08.893868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:08.893881 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:08.894459 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:08.894501 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:08.894520 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:08.894622 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:08.894887 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 11:45:08.895021 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":305,"threads_started":1}
I20260808 11:45:08.896597 27750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:08.977784 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 27573
I20260808 11:45:08.999882 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 27763
I20260808 11:45:09.007966 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:09.000902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51462 (local address 127.13.158.67:45589)
0808 11:45:09.001494 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:09.001501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:09.001531 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:45:09.001591 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:09.001597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:09.001828 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:09.001948 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:09.002351 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:09.003490 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:09.004697 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:09.005230 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:09.005325 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:09.006293 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:09.006301 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:09.006315 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:09.007122 (+ 807us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:09.007176 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:09.007196 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:09.007285 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:09.007416 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 11:45:09.007570 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":299,"threads_started":1}
I20260808 11:45:09.009207 27941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:09.090620 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 27763
I20260808 11:45:09.114969 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 27262
I20260808 11:45:09.116328 27345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:09.217590 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 27262
2026-08-08T11:45:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2640 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T11:45:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:45:09Z Disabled control of system clock
I20260808 11:45:09.303782 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:42615
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:33845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:42615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 11:45:09.637717 27975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:09.638010 27975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:09.638198 27975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:09.645540 27975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:45:09.645637 27975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:09.645710 27975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:09.645808 27975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:45:09.645891 27975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:45:09.655071 27975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:42615
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:42615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:09.656323 27975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:09.658064 27975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:09.669504 27983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:09.669587 27984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:09.669559 27988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:09.670795 27975 server_base.cc:1034] running on GCE node
I20260808 11:45:09.671538 27975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:09.672894 27975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:09.674130 27975 hybrid_clock.cc:648] HybridClock initialized: now 1786189509674102 us; error 42 us; skew 500 ppm
I20260808 11:45:09.674484 27975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:45:09.676868 27975 webserver.cc:460] Webserver started at http://127.13.158.126:33695/ using document root <none> and password file <none>
I20260808 11:45:09.677569 27975 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:09.677682 27975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:09.678046 27975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:09.680909 27975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "1a0e9b6d32cb4b39b56a9383491c1b31"
format_stamp: "Formatted at 2026-08-08 11:45:09 on dist-test-slave-39l8"
server_key: "4c5f233db59b08ea693506a97bbd633c"
server_key_iv: "7912619fc3cb7462261d3787f79a48c9"
server_key_version: "encryptionkey@0"
I20260808 11:45:09.681686 27975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "1a0e9b6d32cb4b39b56a9383491c1b31"
format_stamp: "Formatted at 2026-08-08 11:45:09 on dist-test-slave-39l8"
server_key: "4c5f233db59b08ea693506a97bbd633c"
server_key_iv: "7912619fc3cb7462261d3787f79a48c9"
server_key_version: "encryptionkey@0"
I20260808 11:45:09.686617 27975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260808 11:45:09.689903 27999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:09.691331 27975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:45:09.691504 27975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "1a0e9b6d32cb4b39b56a9383491c1b31"
format_stamp: "Formatted at 2026-08-08 11:45:09 on dist-test-slave-39l8"
server_key: "4c5f233db59b08ea693506a97bbd633c"
server_key_iv: "7912619fc3cb7462261d3787f79a48c9"
server_key_version: "encryptionkey@0"
I20260808 11:45:09.691720 27975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:09.697186 27975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:09.697902 27975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:09.698197 27975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:09.715901 27975 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:42615
I20260808 11:45:09.715934 28063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:42615 every 8 connection(s)
I20260808 11:45:09.717235 27975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:45:09.720955 28064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:45:09.726881 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 27975
I20260808 11:45:09.727119 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:45:09.727762 13945 external_mini_cluster.cc:1428] Setting key 667509179fb122c0431f2c8351974916
I20260808 11:45:09.731917 28064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31: Bootstrap starting.
I20260808 11:45:09.735960 28064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31: Neither blocks nor log segments found. Creating new log.
I20260808 11:45:09.737334 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:09.729881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.13.158.126:42615)
0808 11:45:09.730931 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:09.730946 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:09.730998 (+ 52us) server_negotiation.cc:407] Connection header received
0808 11:45:09.733375 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:09.733401 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:09.733955 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:09.734380 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:09.735410 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:09.735418 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:09.735433 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:09.735450 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:09.735523 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:09.735546 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:09.735735 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:09.735873 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 11:45:09.736151 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":317,"threads_started":1}
I20260808 11:45:09.737329 28064 log.cc:824] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31: Log is configured to *not* fsync() on all Append() calls
I20260808 11:45:09.740748 28064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31: No bootstrap required, opened a new log
I20260808 11:45:09.747906 28064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0e9b6d32cb4b39b56a9383491c1b31" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42615 } }
I20260808 11:45:09.748216 28064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:45:09.748299 28064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0e9b6d32cb4b39b56a9383491c1b31, State: Initialized, Role: FOLLOWER
I20260808 11:45:09.748756 28064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0e9b6d32cb4b39b56a9383491c1b31" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42615 } }
I20260808 11:45:09.748940 28064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:45:09.749028 28064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:45:09.749141 28064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:45:09.751205 28064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0e9b6d32cb4b39b56a9383491c1b31" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42615 } }
I20260808 11:45:09.751734 28064 leader_election.cc:302] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a0e9b6d32cb4b39b56a9383491c1b31; no voters:
I20260808 11:45:09.752496 28064 leader_election.cc:288] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:45:09.752605 28070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:45:09.753296 28070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [term 1 LEADER]: Becoming Leader. State: Replica: 1a0e9b6d32cb4b39b56a9383491c1b31, State: Running, Role: LEADER
I20260808 11:45:09.753819 28070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0e9b6d32cb4b39b56a9383491c1b31" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42615 } }
I20260808 11:45:09.754652 28064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:45:09.756649 28071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a0e9b6d32cb4b39b56a9383491c1b31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0e9b6d32cb4b39b56a9383491c1b31" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42615 } } }
I20260808 11:45:09.757040 28071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [sys.catalog]: This master's current role is: LEADER
I20260808 11:45:09.756942 28072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a0e9b6d32cb4b39b56a9383491c1b31. Latest consensus state: current_term: 1 leader_uuid: "1a0e9b6d32cb4b39b56a9383491c1b31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0e9b6d32cb4b39b56a9383491c1b31" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42615 } } }
I20260808 11:45:09.757364 28072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31 [sys.catalog]: This master's current role is: LEADER
I20260808 11:45:09.760736 28078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:45:09.765898 28078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:45:09.774816 28078 catalog_manager.cc:1347] Generated new cluster ID: 22a792ac4d5141f886d95c83e8f4da48
I20260808 11:45:09.774927 28078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:45:09.788141 28078 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:45:09.788539 28078 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:45:09.794732 28078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a0e9b6d32cb4b39b56a9383491c1b31: Generated new TSK 0
I20260808 11:45:09.795460 28078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:45:09.809561 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:42615
--builtin_ntp_servers=127.13.158.84:33845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 11:45:10.160753 28094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:10.161022 28094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:10.161111 28094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:10.168500 28094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:10.168612 28094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:10.168722 28094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:45:10.177976 28094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:42615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:10.179366 28094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:10.181056 28094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:10.194641 28099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:10.194664 28106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:10.194581 28102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:10.195519 28094 server_base.cc:1034] running on GCE node
I20260808 11:45:10.196257 28094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:10.197396 28094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:10.198608 28094 hybrid_clock.cc:648] HybridClock initialized: now 1786189510198563 us; error 31 us; skew 500 ppm
I20260808 11:45:10.198971 28094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:45:10.201318 28094 webserver.cc:460] Webserver started at http://127.13.158.65:43487/ using document root <none> and password file <none>
I20260808 11:45:10.202010 28094 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:10.202105 28094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:10.202370 28094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:10.207494 28094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "31b47aa8b32e4487b549ddaec6ad5aad"
format_stamp: "Formatted at 2026-08-08 11:45:10 on dist-test-slave-39l8"
server_key: "b448df1ab296953de391149b7b3e2b80"
server_key_iv: "68bbf8285232772995c9d46740b4808d"
server_key_version: "encryptionkey@0"
I20260808 11:45:10.208258 28094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "31b47aa8b32e4487b549ddaec6ad5aad"
format_stamp: "Formatted at 2026-08-08 11:45:10 on dist-test-slave-39l8"
server_key: "b448df1ab296953de391149b7b3e2b80"
server_key_iv: "68bbf8285232772995c9d46740b4808d"
server_key_version: "encryptionkey@0"
I20260808 11:45:10.213757 28094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:45:10.217280 28112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:10.218632 28094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:45:10.218837 28094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "31b47aa8b32e4487b549ddaec6ad5aad"
format_stamp: "Formatted at 2026-08-08 11:45:10 on dist-test-slave-39l8"
server_key: "b448df1ab296953de391149b7b3e2b80"
server_key_iv: "68bbf8285232772995c9d46740b4808d"
server_key_version: "encryptionkey@0"
I20260808 11:45:10.219075 28094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:10.224552 28094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:10.225383 28094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:10.225675 28094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:10.226572 28094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:10.228109 28094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:10.228235 28094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:10.228394 28094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:10.228483 28094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:10.262898 28094 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:33149
I20260808 11:45:10.262943 28273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:33149 every 8 connection(s)
I20260808 11:45:10.264268 28094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:45:10.270268 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 28094
I20260808 11:45:10.270483 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:45:10.271389 13945 external_mini_cluster.cc:1428] Setting key 9e62f53098bcbf17c9bb3eb1511401aa
I20260808 11:45:10.273617 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:10.266963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:47501 (local address 127.13.158.126:42615)
0808 11:45:10.267562 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:10.267566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:10.268663 (+ 1097us) server_negotiation.cc:407] Connection header received
0808 11:45:10.271085 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:10.271090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:10.271172 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:10.271293 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:10.272787 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:10.272792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:10.272795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:10.272811 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:10.272853 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:10.272864 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:10.272962 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:10.273238 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 11:45:10.273302 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":325,"threads_started":1}
I20260808 11:45:10.274653 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:42615
--builtin_ntp_servers=127.13.158.84:33845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 11:45:10.274669 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:10.267607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:42615 (local address 127.13.158.65:47501)
0808 11:45:10.268510 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:10.268544 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:10.270747 (+ 2203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:10.271534 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:10.271561 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:10.271650 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:10.272419 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:45:10.272423 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:10.272476 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:45:10.272590 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:10.272974 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:10.272985 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:10.272993 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:45:10.273275 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 11:45:10.273467 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":270,"threads_started":1}
I20260808 11:45:10.276197 28275 heartbeater.cc:347] Connected to a master server at 127.13.158.126:42615
I20260808 11:45:10.276495 28275 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:10.277048 28275 heartbeater.cc:511] Master 127.13.158.126:42615 requested a full tablet report, sending...
I20260808 11:45:10.279068 28024 ts_manager.cc:194] Registered new tserver with Master: 31b47aa8b32e4487b549ddaec6ad5aad (127.13.158.65:33149)
W20260808 11:45:10.611308 28280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:10.611653 28280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:10.611830 28280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:10.618961 28280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:10.619114 28280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:10.619298 28280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:45:10.628561 28280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:42615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:10.629930 28280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:10.631898 28280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:10.645999 28289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:10.645955 28286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:10.646039 28287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:10.646932 28280 server_base.cc:1034] running on GCE node
I20260808 11:45:10.647553 28280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:10.648509 28280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:10.649736 28280 hybrid_clock.cc:648] HybridClock initialized: now 1786189510649674 us; error 60 us; skew 500 ppm
I20260808 11:45:10.650075 28280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:45:10.652510 28280 webserver.cc:460] Webserver started at http://127.13.158.66:33251/ using document root <none> and password file <none>
I20260808 11:45:10.653226 28280 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:10.653373 28280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:10.653702 28280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:10.656837 28280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "3b4172a7cfaa4347a0c4f61dfa7d0a9a"
format_stamp: "Formatted at 2026-08-08 11:45:10 on dist-test-slave-39l8"
server_key: "a889d79acc0bea940936b0a64fe4a918"
server_key_iv: "683009d3e8a9bd93e185cbeb29f29e01"
server_key_version: "encryptionkey@0"
I20260808 11:45:10.657609 28280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "3b4172a7cfaa4347a0c4f61dfa7d0a9a"
format_stamp: "Formatted at 2026-08-08 11:45:10 on dist-test-slave-39l8"
server_key: "a889d79acc0bea940936b0a64fe4a918"
server_key_iv: "683009d3e8a9bd93e185cbeb29f29e01"
server_key_version: "encryptionkey@0"
I20260808 11:45:10.663029 28280 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:45:10.666466 28297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:10.667925 28280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:45:10.668059 28280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "3b4172a7cfaa4347a0c4f61dfa7d0a9a"
format_stamp: "Formatted at 2026-08-08 11:45:10 on dist-test-slave-39l8"
server_key: "a889d79acc0bea940936b0a64fe4a918"
server_key_iv: "683009d3e8a9bd93e185cbeb29f29e01"
server_key_version: "encryptionkey@0"
I20260808 11:45:10.668284 28280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:10.674078 28280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:10.674927 28280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:10.675158 28280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:10.676013 28280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:10.677341 28280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:10.677428 28280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:10.677510 28280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:10.677561 28280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:10.711853 28280 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:42589
I20260808 11:45:10.711956 28455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:42589 every 8 connection(s)
I20260808 11:45:10.713229 28280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:45:10.721694 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:10.715977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:56723 (local address 127.13.158.126:42615)
0808 11:45:10.716177 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:10.716181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:10.717750 (+ 1569us) server_negotiation.cc:407] Connection header received
0808 11:45:10.719432 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:10.719438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:10.719529 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:10.719650 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:10.720928 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:10.720934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:10.720939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:10.720958 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:10.720997 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:10.721008 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:10.721150 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:10.721336 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 11:45:10.721403 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:45:10.722265 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 28280
I20260808 11:45:10.722460 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:45:10.722941 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:10.716687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:42615 (local address 127.13.158.66:56723)
0808 11:45:10.717592 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:10.717629 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:10.719176 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:10.719871 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:10.719878 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:10.719959 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:10.720649 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:45:10.720651 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:10.720686 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:45:10.720764 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:10.721117 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:10.721130 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:10.721138 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:45:10.721418 (+ 280us) client_negotiation.cc:240] Negotiation successful
0808 11:45:10.721644 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":289,"threads_started":1}
I20260808 11:45:10.723052 13945 external_mini_cluster.cc:1428] Setting key 82a3fdb0e621c0be231c9a8c65ce8332
I20260808 11:45:10.724556 28456 heartbeater.cc:347] Connected to a master server at 127.13.158.126:42615
I20260808 11:45:10.724889 28456 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:10.725474 28456 heartbeater.cc:511] Master 127.13.158.126:42615 requested a full tablet report, sending...
I20260808 11:45:10.725965 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:42615
--builtin_ntp_servers=127.13.158.84:33845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 11:45:10.727497 28024 ts_manager.cc:194] Registered new tserver with Master: 3b4172a7cfaa4347a0c4f61dfa7d0a9a (127.13.158.66:42589)
W20260808 11:45:11.065788 28459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:11.066113 28459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:11.066272 28459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:11.073642 28459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:11.073760 28459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:11.073930 28459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:45:11.083448 28459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:42615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:11.084729 28459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:11.086449 28459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:11.099324 28467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:11.099483 28466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:11.099848 28470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:11.100709 28459 server_base.cc:1034] running on GCE node
I20260808 11:45:11.101287 28459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:11.102223 28459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:11.103441 28459 hybrid_clock.cc:648] HybridClock initialized: now 1786189511103399 us; error 50 us; skew 500 ppm
I20260808 11:45:11.103741 28459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:45:11.105871 28459 webserver.cc:460] Webserver started at http://127.13.158.67:35487/ using document root <none> and password file <none>
I20260808 11:45:11.106547 28459 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:11.106685 28459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:11.107034 28459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:11.109663 28459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "3712119559c74264a0d3e0eafb6f5fed"
format_stamp: "Formatted at 2026-08-08 11:45:11 on dist-test-slave-39l8"
server_key: "f77e8d2acbc42d3cffd1f46809c16193"
server_key_iv: "0b3ab1cf5f26a71fca8f65b6f5c057ed"
server_key_version: "encryptionkey@0"
I20260808 11:45:11.110364 28459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "3712119559c74264a0d3e0eafb6f5fed"
format_stamp: "Formatted at 2026-08-08 11:45:11 on dist-test-slave-39l8"
server_key: "f77e8d2acbc42d3cffd1f46809c16193"
server_key_iv: "0b3ab1cf5f26a71fca8f65b6f5c057ed"
server_key_version: "encryptionkey@0"
I20260808 11:45:11.115335 28459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:45:11.118602 28477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:11.119843 28459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:45:11.119997 28459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "3712119559c74264a0d3e0eafb6f5fed"
format_stamp: "Formatted at 2026-08-08 11:45:11 on dist-test-slave-39l8"
server_key: "f77e8d2acbc42d3cffd1f46809c16193"
server_key_iv: "0b3ab1cf5f26a71fca8f65b6f5c057ed"
server_key_version: "encryptionkey@0"
I20260808 11:45:11.120169 28459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:11.125376 28459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:11.126065 28459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:11.126292 28459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:11.127187 28459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:11.128425 28459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:11.128510 28459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:11.128597 28459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:11.128647 28459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:11.161379 28459 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:34733
I20260808 11:45:11.161418 28635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:34733 every 8 connection(s)
I20260808 11:45:11.162916 28459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:45:11.163806 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 28459
I20260808 11:45:11.163993 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:45:11.164494 13945 external_mini_cluster.cc:1428] Setting key dd54a700e1ee0716d5fbde4223eb4bb9
I20260808 11:45:11.171288 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:11.165585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:60199 (local address 127.13.158.126:42615)
0808 11:45:11.166819 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:11.166830 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:11.167551 (+ 721us) server_negotiation.cc:407] Connection header received
0808 11:45:11.169186 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:11.169189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:11.169251 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:11.169331 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:11.170623 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:11.170627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:11.170630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:11.170650 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:11.170677 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:11.170688 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:11.170868 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:11.171022 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 11:45:11.171071 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029}
I20260808 11:45:11.172291 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:11.166247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:42615 (local address 127.13.158.67:60199)
0808 11:45:11.167395 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:11.167436 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:11.168956 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:11.169559 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:11.169566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:11.169653 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:11.170363 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:45:11.170365 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:11.170402 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:45:11.170484 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:11.170849 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:11.170857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:11.170863 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:45:11.171068 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 11:45:11.171270 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"spinlock_wait_cycles":18816,"thread_start_us":461,"threads_started":1}
I20260808 11:45:11.173606 28637 heartbeater.cc:347] Connected to a master server at 127.13.158.126:42615
I20260808 11:45:11.173898 28637 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:11.174437 28637 heartbeater.cc:511] Master 127.13.158.126:42615 requested a full tablet report, sending...
I20260808 11:45:11.175799 28024 ts_manager.cc:194] Registered new tserver with Master: 3712119559c74264a0d3e0eafb6f5fed (127.13.158.67:34733)
I20260808 11:45:11.179934 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 11:45:11.185716 28278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:45:11.184332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.13.158.126:42615)
0808 11:45:11.184485 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:11.184489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:11.184980 (+ 491us) server_negotiation.cc:407] Connection header received
0808 11:45:11.185104 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:11.185108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:11.185165 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:11.185242 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:11.185516 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45724: BlockingRecv error: recv got EOF from 127.0.0.1:45724 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 11:45:11.186553 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 28094
I20260808 11:45:11.190065 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:11.187596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51976 (local address 127.13.158.65:33149)
0808 11:45:11.188131 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:11.188137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:11.188162 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:45:11.188217 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:11.188223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:11.188402 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:11.188517 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:11.189357 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:11.189363 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:11.189372 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:11.189383 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:11.189419 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:11.189432 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:11.189530 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:11.189649 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 11:45:11.189735 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":283,"threads_started":1}
I20260808 11:45:11.191295 28267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:11.271358 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 28094
I20260808 11:45:11.300716 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 28280
I20260808 11:45:11.304263 28654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:11.301735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51858 (local address 127.13.158.66:42589)
0808 11:45:11.302258 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:11.302265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:11.302291 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:45:11.302358 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:11.302364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:11.302568 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:11.302690 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:11.303645 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:11.303652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:11.303663 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:11.303680 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:11.303730 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:11.303744 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:11.303838 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:11.303950 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 11:45:11.304030 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":250,"threads_started":1}
I20260808 11:45:11.305532 28448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:11.384335 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 28280
I20260808 11:45:11.409404 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 28459
I20260808 11:45:11.413159 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:11.410436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.13.158.67:34733)
0808 11:45:11.411077 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:11.411092 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:11.411122 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:45:11.411184 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:11.411191 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:11.411406 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:11.411533 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:11.412556 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:11.412561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:11.412572 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:45:11.412583 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:11.412625 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:45:11.412638 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:11.412777 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:11.412828 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 11:45:11.412904 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":335,"threads_started":1}
I20260808 11:45:11.414426 28629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:11.493080 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 28459
I20260808 11:45:11.517462 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 27975
I20260808 11:45:11.518926 28058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:11.620198 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 27975
2026-08-08T11:45:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2402 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:45:11 dist-test-slave-39l8 krb5kdc[28663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:45:11 dist-test-slave-39l8 krb5kdc[28663](info): set up 2 sockets
Aug 08 11:45:11 dist-test-slave-39l8 krb5kdc[28663](info): commencing operation
krb5kdc: starting...
W20260808 11:45:12.894097 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:45:13 dist-test-slave-39l8 krb5kdc[28663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:45:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:45:13.154592 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:33945
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:33945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:45:13.481566 28679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:13.481842 28679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:13.482000 28679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:13.489039 28679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:45:13.489136 28679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:13.489276 28679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:13.489370 28679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:45:13.489435 28679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:45:13.498886 28679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:33945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:33945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:13.500169 28679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:13.501418 28679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:13.513063 28684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:13.513113 28685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:13.513278 28689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:13.514201 28679 server_base.cc:1034] running on GCE node
I20260808 11:45:13.515043 28679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:13.516337 28679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:13.517560 28679 hybrid_clock.cc:648] HybridClock initialized: now 1786189513517513 us; error 35 us; skew 500 ppm
I20260808 11:45:13.517899 28679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:45:13 dist-test-slave-39l8 krb5kdc[28663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189513, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:45:13.525345 28679 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:45:13.526950 28679 webserver.cc:460] Webserver started at http://127.13.158.126:41245/ using document root <none> and password file <none>
I20260808 11:45:13.527661 28679 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:13.527796 28679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:13.528143 28679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:13.531111 28679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "7842d1b67ecf41589d1b6dc87b36ccf1"
format_stamp: "Formatted at 2026-08-08 11:45:13 on dist-test-slave-39l8"
server_key: "133247ada562982c24fe3114d966531b"
server_key_iv: "e07f917d59431a9066a36f973e276e08"
server_key_version: "encryptionkey@0"
I20260808 11:45:13.531862 28679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "7842d1b67ecf41589d1b6dc87b36ccf1"
format_stamp: "Formatted at 2026-08-08 11:45:13 on dist-test-slave-39l8"
server_key: "133247ada562982c24fe3114d966531b"
server_key_iv: "e07f917d59431a9066a36f973e276e08"
server_key_version: "encryptionkey@0"
I20260808 11:45:13.536924 28679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:45:13.540341 28699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:13.541579 28679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:45:13.541760 28679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "7842d1b67ecf41589d1b6dc87b36ccf1"
format_stamp: "Formatted at 2026-08-08 11:45:13 on dist-test-slave-39l8"
server_key: "133247ada562982c24fe3114d966531b"
server_key_iv: "e07f917d59431a9066a36f973e276e08"
server_key_version: "encryptionkey@0"
I20260808 11:45:13.541949 28679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:13.563213 28679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:13.567155 28679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:13.567426 28679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:13.586207 28679 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:33945
I20260808 11:45:13.586237 28761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:33945 every 8 connection(s)
I20260808 11:45:13.587625 28679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:45:13.589573 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 28679
I20260808 11:45:13.589828 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:45:13.590682 13945 external_mini_cluster.cc:1428] Setting key 39186d878f48b2060ed41b3ef34c7931
I20260808 11:45:13.592545 28763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:45:13 dist-test-slave-39l8 krb5kdc[28663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:45:13.606474 28763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1: Bootstrap starting.
I20260808 11:45:13.609759 28763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1: Neither blocks nor log segments found. Creating new log.
I20260808 11:45:13.610829 28763 log.cc:824] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1: Log is configured to *not* fsync() on all Append() calls
I20260808 11:45:13.613499 28767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:13.592816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43476 (local address 127.13.158.126:33945)
0808 11:45:13.593603 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:13.593613 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:13.593651 (+ 38us) server_negotiation.cc:407] Connection header received
0808 11:45:13.595305 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:13.595326 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:13.595702 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:13.596058 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:13.597878 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:13.599288 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:13.600997 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:13.601605 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:13.601727 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:13.606509 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:13.606517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:13.606530 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:13.606584 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:13.609510 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:13.610247 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:13.610253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:13.610259 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:13.610409 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:13.610819 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:13.610825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:13.610834 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:13.611412 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:13.611744 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:13.612171 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 11:45:13.612568 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":393,"threads_started":1}
I20260808 11:45:13.613754 28763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1: No bootstrap required, opened a new log
I20260808 11:45:13.623888 28763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7842d1b67ecf41589d1b6dc87b36ccf1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33945 } }
I20260808 11:45:13.624365 28763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:45:13.624460 28763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7842d1b67ecf41589d1b6dc87b36ccf1, State: Initialized, Role: FOLLOWER
I20260808 11:45:13.625103 28763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7842d1b67ecf41589d1b6dc87b36ccf1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33945 } }
I20260808 11:45:13.625418 28763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:45:13.625554 28763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:45:13.625744 28763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:45:13.628695 28763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7842d1b67ecf41589d1b6dc87b36ccf1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33945 } }
I20260808 11:45:13.629439 28763 leader_election.cc:302] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7842d1b67ecf41589d1b6dc87b36ccf1; no voters:
I20260808 11:45:13.630010 28763 leader_election.cc:288] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:45:13.630182 28770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:45:13.630978 28770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [term 1 LEADER]: Becoming Leader. State: Replica: 7842d1b67ecf41589d1b6dc87b36ccf1, State: Running, Role: LEADER
I20260808 11:45:13.631549 28770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7842d1b67ecf41589d1b6dc87b36ccf1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33945 } }
I20260808 11:45:13.632416 28763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:45:13.634407 28773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7842d1b67ecf41589d1b6dc87b36ccf1. Latest consensus state: current_term: 1 leader_uuid: "7842d1b67ecf41589d1b6dc87b36ccf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7842d1b67ecf41589d1b6dc87b36ccf1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33945 } } }
I20260808 11:45:13.634716 28773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [sys.catalog]: This master's current role is: LEADER
I20260808 11:45:13.635609 28777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:45:13.635473 28771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7842d1b67ecf41589d1b6dc87b36ccf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7842d1b67ecf41589d1b6dc87b36ccf1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33945 } } }
I20260808 11:45:13.635821 28771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1 [sys.catalog]: This master's current role is: LEADER
I20260808 11:45:13.640774 28777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:45:13.650856 28777 catalog_manager.cc:1347] Generated new cluster ID: e1b95e2650a1402a874ec7f32225425c
I20260808 11:45:13.650969 28777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:45:13.663440 28777 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:45:13.664783 28777 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:45:13.671734 28777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1: Generated new TSK 0
I20260808 11:45:13.672487 28777 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:45:13.738504 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33945
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:45:14.080925 28802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:14.081236 28802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:14.081415 28802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:14.088515 28802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:14.088689 28802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:14.088881 28802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:45:14.098047 28802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:14.099324 28802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:14.100596 28802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:14.113761 28807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:14.114310 28808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:14.114466 28810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:14.115482 28802 server_base.cc:1034] running on GCE node
I20260808 11:45:14.116086 28802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:14.116997 28802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:14.118198 28802 hybrid_clock.cc:648] HybridClock initialized: now 1786189514118153 us; error 36 us; skew 500 ppm
I20260808 11:45:14.118520 28802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:45:14 dist-test-slave-39l8 krb5kdc[28663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:45:14.121564 28802 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:45:14.123113 28802 webserver.cc:460] Webserver started at http://127.13.158.65:36019/ using document root <none> and password file <none>
I20260808 11:45:14.123816 28802 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:14.123962 28802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:14.124281 28802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:14.127241 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "e194b41465724527ac498242f9acde95"
format_stamp: "Formatted at 2026-08-08 11:45:14 on dist-test-slave-39l8"
server_key: "05539674b0cc6aff12b83499a063d642"
server_key_iv: "4768cd19a23873ab0563fc8535a6af06"
server_key_version: "encryptionkey@0"
I20260808 11:45:14.128005 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "e194b41465724527ac498242f9acde95"
format_stamp: "Formatted at 2026-08-08 11:45:14 on dist-test-slave-39l8"
server_key: "05539674b0cc6aff12b83499a063d642"
server_key_iv: "4768cd19a23873ab0563fc8535a6af06"
server_key_version: "encryptionkey@0"
I20260808 11:45:14.133080 28802 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 11:45:14.136622 28821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:14.137882 28802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:45:14.138047 28802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "e194b41465724527ac498242f9acde95"
format_stamp: "Formatted at 2026-08-08 11:45:14 on dist-test-slave-39l8"
server_key: "05539674b0cc6aff12b83499a063d642"
server_key_iv: "4768cd19a23873ab0563fc8535a6af06"
server_key_version: "encryptionkey@0"
I20260808 11:45:14.138240 28802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:14.152940 28802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:14.156620 28802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:14.156919 28802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:14.157812 28802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:14.159194 28802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:14.159297 28802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:14.159441 28802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:14.159526 28802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:14.193378 28802 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:35287
I20260808 11:45:14.193420 28967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:35287 every 8 connection(s)
I20260808 11:45:14.194691 28802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:45:14.205214 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 28802
I20260808 11:45:14.205457 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:45:14.206343 13945 external_mini_cluster.cc:1428] Setting key 2f79bc5e9ae640d538921eb38a49fc68
Aug 08 11:45:14 dist-test-slave-39l8 krb5kdc[28663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:45:14.214967 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:14.197281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:48785 (local address 127.13.158.126:33945)
0808 11:45:14.197807 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:14.197813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:14.198762 (+ 949us) server_negotiation.cc:407] Connection header received
0808 11:45:14.200460 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:14.200464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:14.200528 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:14.200620 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:14.201979 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:14.202657 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:14.204191 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:14.204612 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:14.204729 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:14.209149 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:14.209154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:14.209156 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:14.209186 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:14.211809 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:14.212713 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:14.212717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:14.212719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:14.212828 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:14.213307 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:14.213310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:14.213312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:14.213555 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:14.213661 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:14.214499 (+ 838us) server_negotiation.cc:299] Negotiation successful
0808 11:45:14.214653 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":254,"threads_started":1}
I20260808 11:45:14.216506 28976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:14.197869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33945 (local address 127.13.158.65:48785)
0808 11:45:14.198565 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:14.198626 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:14.200224 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:14.200820 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:14.200827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:14.201305 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:14.201819 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:14.201835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:14.202871 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:14.202875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:14.204063 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:14.204075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:14.204869 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:14.204876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:14.205195 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:14.205927 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:14.205959 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:14.208941 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:14.211950 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:45:14.211961 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:45:14.211978 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:45:14.212592 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:45:14.212936 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:45:14.212943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:45:14.212949 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:45:14.213201 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:45:14.213684 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:14.213694 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:14.214124 (+ 430us) client_negotiation.cc:769] Sending connection context
0808 11:45:14.214438 (+ 314us) client_negotiation.cc:240] Negotiation successful
0808 11:45:14.215046 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":267,"threads_started":1}
I20260808 11:45:14.218099 28969 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33945
I20260808 11:45:14.218393 28969 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:14.219146 28969 heartbeater.cc:511] Master 127.13.158.126:33945 requested a full tablet report, sending...
I20260808 11:45:14.221319 28721 ts_manager.cc:194] Registered new tserver with Master: e194b41465724527ac498242f9acde95 (127.13.158.65:35287)
I20260808 11:45:14.222745 28721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:48785
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:45:14.266394 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33945
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:45:14.601420 28982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:14.601781 28982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:14.601923 28982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:14.612516 28982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:14.612648 28982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:14.612766 28982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:45:14.623399 28982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:14.624687 28982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:14.625962 28982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:14.639226 28988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:14.639452 28989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:14.639269 28991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:14.640420 28982 server_base.cc:1034] running on GCE node
I20260808 11:45:14.641089 28982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:14.641983 28982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:14.643208 28982 hybrid_clock.cc:648] HybridClock initialized: now 1786189514643170 us; error 62 us; skew 500 ppm
I20260808 11:45:14.643533 28982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:45:14 dist-test-slave-39l8 krb5kdc[28663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:45:14.646983 28982 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:45:14.648412 28982 webserver.cc:460] Webserver started at http://127.13.158.66:35341/ using document root <none> and password file <none>
I20260808 11:45:14.649117 28982 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:14.649266 28982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:14.649583 28982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:14.652496 28982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "f9ad408e40ad4f9aa17dfe024159dc71"
format_stamp: "Formatted at 2026-08-08 11:45:14 on dist-test-slave-39l8"
server_key: "ffe24d29004eb2c2fd5b45fecb32fb91"
server_key_iv: "ee15e41abcf06320d260619c59b59349"
server_key_version: "encryptionkey@0"
I20260808 11:45:14.653249 28982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "f9ad408e40ad4f9aa17dfe024159dc71"
format_stamp: "Formatted at 2026-08-08 11:45:14 on dist-test-slave-39l8"
server_key: "ffe24d29004eb2c2fd5b45fecb32fb91"
server_key_iv: "ee15e41abcf06320d260619c59b59349"
server_key_version: "encryptionkey@0"
I20260808 11:45:14.658190 28982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:45:14.661473 29000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:14.662796 28982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:45:14.662957 28982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "f9ad408e40ad4f9aa17dfe024159dc71"
format_stamp: "Formatted at 2026-08-08 11:45:14 on dist-test-slave-39l8"
server_key: "ffe24d29004eb2c2fd5b45fecb32fb91"
server_key_iv: "ee15e41abcf06320d260619c59b59349"
server_key_version: "encryptionkey@0"
I20260808 11:45:14.663168 28982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:14.688496 28982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:14.692521 28982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:14.692828 28982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:14.693950 28982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:14.695448 28982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:14.695556 28982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:14.695712 28982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:14.695791 28982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:14.728966 28982 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:33259
I20260808 11:45:14.729010 29159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:33259 every 8 connection(s)
I20260808 11:45:14.730242 28982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:45:14.732656 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 28982
I20260808 11:45:14.732920 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:45:14.733357 13945 external_mini_cluster.cc:1428] Setting key d5c867032a6498e8d7716fd4e118d1bb
Aug 08 11:45:14 dist-test-slave-39l8 krb5kdc[28663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:45:14.752100 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:14.732781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:49381 (local address 127.13.158.126:33945)
0808 11:45:14.735077 (+ 2296us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:14.735082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:14.735103 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:45:14.736669 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:14.736674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:14.736774 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:14.736891 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:14.738384 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:14.739202 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:14.740700 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:14.741105 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:14.741249 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:14.745566 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:14.745571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:14.745574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:14.745606 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:14.748815 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:14.749734 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:14.749738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:14.749740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:14.749869 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:14.750343 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:14.750346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:14.750348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:14.750603 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:14.750775 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:14.751592 (+ 817us) server_negotiation.cc:299] Negotiation successful
0808 11:45:14.751760 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2173,"thread_start_us":2071,"threads_started":1}
I20260808 11:45:14.753664 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:14.733372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33945 (local address 127.13.158.66:49381)
0808 11:45:14.734478 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:14.734516 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:14.736414 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:14.737131 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:14.737139 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:14.737651 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:14.738221 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:14.738237 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:14.739354 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:14.739362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:14.740541 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:14.740558 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:14.741421 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:14.741431 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:14.741779 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:14.742508 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:14.742541 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:14.745349 (+ 2808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:14.748970 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:45:14.748982 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:45:14.749000 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:45:14.749608 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:45:14.749982 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:45:14.749989 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:45:14.749996 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:45:14.750236 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:45:14.750757 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:14.750767 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:14.751198 (+ 431us) client_negotiation.cc:769] Sending connection context
0808 11:45:14.751525 (+ 327us) client_negotiation.cc:240] Negotiation successful
0808 11:45:14.752147 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":573,"threads_started":1}
I20260808 11:45:14.755129 29160 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33945
I20260808 11:45:14.755482 29160 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:14.756224 29160 heartbeater.cc:511] Master 127.13.158.126:33945 requested a full tablet report, sending...
I20260808 11:45:14.757733 28721 ts_manager.cc:194] Registered new tserver with Master: f9ad408e40ad4f9aa17dfe024159dc71 (127.13.158.66:33259)
I20260808 11:45:14.758555 28721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:49381
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:45:14.797549 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33945
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:45:15.131361 29170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:15.131672 29170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:15.131844 29170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:15.138975 29170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:15.139128 29170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:15.139305 29170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:45:15.149240 29170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:15.150480 29170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:15.151834 29170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:15.165272 29179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:15.165225 29180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:15.165184 29185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:15.166164 29170 server_base.cc:1034] running on GCE node
I20260808 11:45:15.166864 29170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:15.167888 29170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:15.169138 29170 hybrid_clock.cc:648] HybridClock initialized: now 1786189515169072 us; error 56 us; skew 500 ppm
I20260808 11:45:15.169456 29170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:45:15 dist-test-slave-39l8 krb5kdc[28663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189515, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:45:15.172650 29170 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:45:15.174089 29170 webserver.cc:460] Webserver started at http://127.13.158.67:39055/ using document root <none> and password file <none>
I20260808 11:45:15.174809 29170 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:15.174912 29170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:15.175163 29170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:45:15.178020 29170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "c506a6aa75ae4c21a8a9fdac3378bd24"
format_stamp: "Formatted at 2026-08-08 11:45:15 on dist-test-slave-39l8"
server_key: "9e753e18a9b7a71bd5e1952f51a8d70e"
server_key_iv: "97327391656fd8758e4d28b487b717ab"
server_key_version: "encryptionkey@0"
I20260808 11:45:15.178750 29170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "c506a6aa75ae4c21a8a9fdac3378bd24"
format_stamp: "Formatted at 2026-08-08 11:45:15 on dist-test-slave-39l8"
server_key: "9e753e18a9b7a71bd5e1952f51a8d70e"
server_key_iv: "97327391656fd8758e4d28b487b717ab"
server_key_version: "encryptionkey@0"
I20260808 11:45:15.183905 29170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:45:15.187268 29194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:15.188546 29170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260808 11:45:15.188720 29170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "c506a6aa75ae4c21a8a9fdac3378bd24"
format_stamp: "Formatted at 2026-08-08 11:45:15 on dist-test-slave-39l8"
server_key: "9e753e18a9b7a71bd5e1952f51a8d70e"
server_key_iv: "97327391656fd8758e4d28b487b717ab"
server_key_version: "encryptionkey@0"
I20260808 11:45:15.188905 29170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:45:15.207913 29170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:45:15.211812 29170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:45:15.212116 29170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:45:15.213059 29170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:45:15.214349 29170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:45:15.214421 29170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:15.214592 29170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:45:15.214643 29170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:45:15.226352 28969 heartbeater.cc:503] Master 127.13.158.126:33945 was elected leader, sending a full tablet report...
I20260808 11:45:15.248234 29170 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:37583
I20260808 11:45:15.248270 29348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:37583 every 8 connection(s)
I20260808 11:45:15.249558 29170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:45:15.253623 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 29170
I20260808 11:45:15.253844 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:45:15.254266 13945 external_mini_cluster.cc:1428] Setting key b45f1432839d8d31ffcbbf057b82fd24
Aug 08 11:45:15 dist-test-slave-39l8 krb5kdc[28663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189515, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:45:15.269157 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.252033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:34065 (local address 127.13.158.126:33945)
0808 11:45:15.252240 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.252244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.253225 (+ 981us) server_negotiation.cc:407] Connection header received
0808 11:45:15.255373 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.255378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.255506 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.255619 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:15.257066 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.257810 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.259500 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.259895 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.260031 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.263382 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:15.263386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:15.263390 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:15.263422 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:15.265975 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.266741 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.266748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.266751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.266880 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.267334 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.267339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.267342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.267639 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:15.267834 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.268576 (+ 742us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.268816 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 11:45:15.270181 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.252370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33945 (local address 127.13.158.67:34065)
0808 11:45:15.253056 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:15.253101 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:15.255053 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:15.255857 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:15.255866 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:15.256356 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:45:15.256897 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.256913 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.258024 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.258030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.259348 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.259360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.260119 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.260127 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.260376 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.261006 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:45:15.261033 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:45:15.263163 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:45:15.266121 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:45:15.266129 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:45:15.266151 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:45:15.266556 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:45:15.267017 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:45:15.267025 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:45:15.267028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:45:15.267203 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:45:15.267789 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:45:15.267798 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:45:15.268167 (+ 369us) client_negotiation.cc:769] Sending connection context
0808 11:45:15.268446 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 11:45:15.268973 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":265,"threads_started":1}
I20260808 11:45:15.271601 29349 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33945
I20260808 11:45:15.271884 29349 heartbeater.cc:464] Registering TS with master...
I20260808 11:45:15.272588 29349 heartbeater.cc:511] Master 127.13.158.126:33945 requested a full tablet report, sending...
I20260808 11:45:15.273932 28721 ts_manager.cc:194] Registered new tserver with Master: c506a6aa75ae4c21a8a9fdac3378bd24 (127.13.158.67:37583)
I20260808 11:45:15.274688 28721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:34065
I20260808 11:45:15.281930 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:45:15.298779 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.286498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43480 (local address 127.13.158.126:33945)
0808 11:45:15.286667 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.286673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.287256 (+ 583us) server_negotiation.cc:407] Connection header received
0808 11:45:15.287383 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.287386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.287451 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.287529 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:15.288898 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.289592 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.290782 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.291168 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.291257 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.293709 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:15.293715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:15.293722 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:15.293766 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:15.296414 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.296958 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.296961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.296963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.297072 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.297399 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.297402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.297404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.297692 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:15.297890 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.298113 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.298332 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:45:15.302502 28721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:45:15.309202 28721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:45:15.346112 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.321482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35598 (local address 127.13.158.66:33259)
0808 11:45:15.335697 (+ 14215us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.335709 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.335750 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:45:15.336284 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.336297 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.336596 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.336816 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:45:15.338343 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.340430 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.343375 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.344823 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.345059 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.345324 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.345580 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.345746 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14031,"thread_start_us":13919,"threads_started":1}
I20260808 11:45:15.346268 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.320851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:33259 (local address 127.0.0.1:35598)
0808 11:45:15.335654 (+ 14803us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:15.335678 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:15.335821 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:15.337163 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:15.337170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:15.337225 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:45:15.338108 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.338121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.340652 (+ 2531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.340660 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.343130 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.343145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.345111 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.345117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.345311 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.345397 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 11:45:15.345538 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 11:45:15.345784 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14626,"mutex_wait_us":60,"thread_start_us":7441,"threads_started":1}
I20260808 11:45:15.347121 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.321000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.13.158.67:37583)
0808 11:45:15.336219 (+ 15219us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.336230 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.336269 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:45:15.336385 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.336395 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.336680 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.337579 (+ 899us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:45:15.337911 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.339832 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.343307 (+ 3475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.344870 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.345063 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.345328 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.346013 (+ 685us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.346793 (+ 780us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14992,"thread_start_us":15495,"threads_started":1}
I20260808 11:45:15.347124 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.320676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37583 (local address 127.0.0.1:41904)
0808 11:45:15.328673 (+ 7997us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:15.328723 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:15.328918 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:15.336985 (+ 8067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:15.336996 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:15.337038 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:45:15.337685 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.337702 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.340028 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.340036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.343011 (+ 2975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.343037 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.345474 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.345480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.345752 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.345786 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:45:15.345892 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 11:45:15.346652 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7741,"mutex_wait_us":702,"thread_start_us":15248,"threads_started":1}
I20260808 11:45:15.350694 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.330432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:35287 (local address 127.0.0.1:39098)
0808 11:45:15.341595 (+ 11163us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:15.341612 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:15.341740 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:15.342770 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:15.342774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:15.342799 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:45:15.343341 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.343355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.345286 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.345293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.347963 (+ 2670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.347976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.349976 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.349982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.350151 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.350180 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 11:45:15.350256 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 11:45:15.350366 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11027,"mutex_wait_us":33,"thread_start_us":10858,"threads_started":1}
I20260808 11:45:15.350335 29073 tablet_service.cc:1467] Processing CreateTablet for tablet f98e05e42fd549f79be76b9b2edb1745 (DEFAULT_TABLE table=test-table [id=fec66f10ad3e424bb0944f023ed043c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:45:15.350915 29261 tablet_service.cc:1467] Processing CreateTablet for tablet f98e05e42fd549f79be76b9b2edb1745 (DEFAULT_TABLE table=test-table [id=fec66f10ad3e424bb0944f023ed043c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:45:15.351804 29073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98e05e42fd549f79be76b9b2edb1745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:45:15.351853 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.330861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.13.158.65:35287)
0808 11:45:15.342121 (+ 11260us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.342130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.342165 (+ 35us) server_negotiation.cc:407] Connection header received
0808 11:45:15.342254 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.342260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.342495 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.342760 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:45:15.343517 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.345122 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.348113 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.349490 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.349620 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.349849 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.351282 (+ 1433us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.351441 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11035,"thread_start_us":10964,"threads_started":1}
I20260808 11:45:15.352355 29261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98e05e42fd549f79be76b9b2edb1745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:45:15.355511 28884 tablet_service.cc:1467] Processing CreateTablet for tablet f98e05e42fd549f79be76b9b2edb1745 (DEFAULT_TABLE table=test-table [id=fec66f10ad3e424bb0944f023ed043c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:45:15.356854 28884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f98e05e42fd549f79be76b9b2edb1745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:45:15.363270 29379 tablet_bootstrap.cc:492] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: Bootstrap starting.
I20260808 11:45:15.366106 29380 tablet_bootstrap.cc:492] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: Bootstrap starting.
I20260808 11:45:15.366871 29379 tablet_bootstrap.cc:654] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: Neither blocks nor log segments found. Creating new log.
I20260808 11:45:15.368320 29379 log.cc:824] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: Log is configured to *not* fsync() on all Append() calls
I20260808 11:45:15.369626 29380 tablet_bootstrap.cc:654] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: Neither blocks nor log segments found. Creating new log.
I20260808 11:45:15.370802 29380 log.cc:824] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: Log is configured to *not* fsync() on all Append() calls
I20260808 11:45:15.371832 29379 tablet_bootstrap.cc:492] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: No bootstrap required, opened a new log
I20260808 11:45:15.371871 29383 tablet_bootstrap.cc:492] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Bootstrap starting.
I20260808 11:45:15.372202 29379 ts_tablet_manager.cc:1397] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260808 11:45:15.374854 29383 tablet_bootstrap.cc:654] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Neither blocks nor log segments found. Creating new log.
I20260808 11:45:15.375780 29383 log.cc:824] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Log is configured to *not* fsync() on all Append() calls
I20260808 11:45:15.376806 29380 tablet_bootstrap.cc:492] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: No bootstrap required, opened a new log
I20260808 11:45:15.377070 29380 ts_tablet_manager.cc:1397] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260808 11:45:15.378575 29383 tablet_bootstrap.cc:492] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: No bootstrap required, opened a new log
I20260808 11:45:15.378880 29383 ts_tablet_manager.cc:1397] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260808 11:45:15.383121 29379 raft_consensus.cc:348] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.383628 29379 raft_consensus.cc:374] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:45:15.383728 29379 raft_consensus.cc:729] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c506a6aa75ae4c21a8a9fdac3378bd24, State: Initialized, Role: FOLLOWER
I20260808 11:45:15.384475 29379 consensus_queue.cc:260] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.385789 29349 heartbeater.cc:503] Master 127.13.158.126:33945 was elected leader, sending a full tablet report...
I20260808 11:45:15.386257 29379 ts_tablet_manager.cc:1428] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260808 11:45:15.387343 29383 raft_consensus.cc:348] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.387332 29380 raft_consensus.cc:348] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.387725 29383 raft_consensus.cc:374] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:45:15.387738 29380 raft_consensus.cc:374] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:45:15.387820 29383 raft_consensus.cc:729] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e194b41465724527ac498242f9acde95, State: Initialized, Role: FOLLOWER
I20260808 11:45:15.387862 29380 raft_consensus.cc:729] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ad408e40ad4f9aa17dfe024159dc71, State: Initialized, Role: FOLLOWER
I20260808 11:45:15.388377 29383 consensus_queue.cc:260] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.388454 29380 consensus_queue.cc:260] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.390117 29383 ts_tablet_manager.cc:1428] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260808 11:45:15.391392 29160 heartbeater.cc:503] Master 127.13.158.126:33945 was elected leader, sending a full tablet report...
I20260808 11:45:15.391906 29380 ts_tablet_manager.cc:1428] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260808 11:45:15.413735 29392 raft_consensus.cc:482] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:45:15.413964 29392 raft_consensus.cc:504] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.415596 29392 leader_election.cc:288] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c506a6aa75ae4c21a8a9fdac3378bd24 (127.13.158.67:37583), f9ad408e40ad4f9aa17dfe024159dc71 (127.13.158.66:33259)
I20260808 11:45:15.423053 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.415938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:43997 (local address 127.13.158.67:37583)
0808 11:45:15.416415 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.416427 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.416963 (+ 536us) server_negotiation.cc:407] Connection header received
0808 11:45:15.417190 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.417197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.417286 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.417383 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:45:15.418087 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.419092 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.421380 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.422093 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.422240 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.422323 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.422638 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.422787 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324}
I20260808 11:45:15.423064 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.416187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:44601 (local address 127.13.158.66:33259)
0808 11:45:15.416454 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.416458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.417335 (+ 877us) server_negotiation.cc:407] Connection header received
0808 11:45:15.417497 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.417501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.417582 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.417694 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:45:15.418334 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.419191 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.421374 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.422124 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.422301 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.422369 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.422679 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.422814 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260808 11:45:15.423348 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.415809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37583 (local address 127.13.158.65:43997)
0808 11:45:15.416833 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:15.416859 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:15.417028 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:15.417446 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:15.417451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:15.417481 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:45:15.417939 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.417951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.419232 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.419240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.421226 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.421238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.422302 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.422308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.422482 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.422505 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:45:15.422579 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 11:45:15.422753 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":125,"thread_start_us":392,"threads_started":1}
I20260808 11:45:15.423409 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.416149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:33259 (local address 127.13.158.65:44601)
0808 11:45:15.417219 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0808 11:45:15.417237 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:45:15.417366 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:45:15.417718 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:45:15.417721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:45:15.417742 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:45:15.418132 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.418148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.419343 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.419347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.421197 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:45:15.421214 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.422391 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.422396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:45:15.422499 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.422520 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 11:45:15.422626 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 11:45:15.422771 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":53,"thread_start_us":688,"threads_started":1}
I20260808 11:45:15.424522 29099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98e05e42fd549f79be76b9b2edb1745" candidate_uuid: "e194b41465724527ac498242f9acde95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" is_pre_election: true
I20260808 11:45:15.424520 29284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98e05e42fd549f79be76b9b2edb1745" candidate_uuid: "e194b41465724527ac498242f9acde95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" is_pre_election: true
I20260808 11:45:15.425071 29284 raft_consensus.cc:2454] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e194b41465724527ac498242f9acde95 in term 0.
I20260808 11:45:15.425071 29099 raft_consensus.cc:2454] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e194b41465724527ac498242f9acde95 in term 0.
I20260808 11:45:15.425827 28825 leader_election.cc:302] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c506a6aa75ae4c21a8a9fdac3378bd24, e194b41465724527ac498242f9acde95; no voters:
I20260808 11:45:15.426304 29392 raft_consensus.cc:2781] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:45:15.426461 29392 raft_consensus.cc:482] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:45:15.426570 29392 raft_consensus.cc:3037] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:45:15.428791 29392 raft_consensus.cc:504] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.429373 29392 leader_election.cc:288] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [CANDIDATE]: Term 1 election: Requested vote from peers c506a6aa75ae4c21a8a9fdac3378bd24 (127.13.158.67:37583), f9ad408e40ad4f9aa17dfe024159dc71 (127.13.158.66:33259)
I20260808 11:45:15.429795 29284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98e05e42fd549f79be76b9b2edb1745" candidate_uuid: "e194b41465724527ac498242f9acde95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24"
I20260808 11:45:15.429807 29099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f98e05e42fd549f79be76b9b2edb1745" candidate_uuid: "e194b41465724527ac498242f9acde95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ad408e40ad4f9aa17dfe024159dc71"
I20260808 11:45:15.429992 29284 raft_consensus.cc:3037] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:45:15.430001 29099 raft_consensus.cc:3037] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:45:15.432209 29284 raft_consensus.cc:2454] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e194b41465724527ac498242f9acde95 in term 1.
I20260808 11:45:15.432238 29099 raft_consensus.cc:2454] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e194b41465724527ac498242f9acde95 in term 1.
I20260808 11:45:15.432683 28824 leader_election.cc:302] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e194b41465724527ac498242f9acde95, f9ad408e40ad4f9aa17dfe024159dc71; no voters:
I20260808 11:45:15.433002 29392 raft_consensus.cc:2781] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:45:15.433525 29392 raft_consensus.cc:686] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 1 LEADER]: Becoming Leader. State: Replica: e194b41465724527ac498242f9acde95, State: Running, Role: LEADER
I20260808 11:45:15.434041 29392 consensus_queue.cc:237] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } }
I20260808 11:45:15.438355 28719 catalog_manager.cc:5495] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 reported cstate change: term changed from 0 to 1, leader changed from <none> to e194b41465724527ac498242f9acde95 (127.13.158.65). New cstate: current_term: 1 leader_uuid: "e194b41465724527ac498242f9acde95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e194b41465724527ac498242f9acde95" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 35287 } health_report { overall_health: HEALTHY } } }
W20260808 11:45:15.447033 28972 tablet.cc:2369] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:45:15.455965 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.450564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.13.158.65:35287)
0808 11:45:15.450810 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.450816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.450919 (+ 103us) server_negotiation.cc:407] Connection header received
0808 11:45:15.451162 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.451165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.451256 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.451395 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:45:15.451848 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.452571 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.453878 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.454270 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.454400 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.454890 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:45:15.455345 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:45:15.455493 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.455596 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.455673 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 11:45:15.464121 29099 raft_consensus.cc:1261] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 1 FOLLOWER]: Refusing update from remote peer e194b41465724527ac498242f9acde95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:45:15.464123 29284 raft_consensus.cc:1261] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 1 FOLLOWER]: Refusing update from remote peer e194b41465724527ac498242f9acde95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:45:15.465229 29399 consensus_queue.cc:1035] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c506a6aa75ae4c21a8a9fdac3378bd24" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:45:15.465575 29392 consensus_queue.cc:1035] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ad408e40ad4f9aa17dfe024159dc71" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:45:15.491349 28719 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:45:15.491662 28719 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:45:15.495175 28719 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7842d1b67ecf41589d1b6dc87b36ccf1: Sending DeleteTablet for 3 replicas of tablet f98e05e42fd549f79be76b9b2edb1745
I20260808 11:45:15.496321 29261 tablet_service.cc:1514] Processing DeleteTablet for tablet f98e05e42fd549f79be76b9b2edb1745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:45:15 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:41904
I20260808 11:45:15.496631 28884 tablet_service.cc:1514] Processing DeleteTablet for tablet f98e05e42fd549f79be76b9b2edb1745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:45:15 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:39098
I20260808 11:45:15.496657 29073 tablet_service.cc:1514] Processing DeleteTablet for tablet f98e05e42fd549f79be76b9b2edb1745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:45:15 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:35598
I20260808 11:45:15.498898 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 28802
I20260808 11:45:15.512190 29419 tablet_replica.cc:330] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: stopping tablet replica
I20260808 11:45:15.512701 29419 raft_consensus.cc:2229] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:45:15.513295 29419 raft_consensus.cc:2258] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:45:15.515892 29419 ts_tablet_manager.cc:1905] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:45:15.524410 29419 ts_tablet_manager.cc:1918] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:45:15.524583 29419 log.cc:1196] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/f98e05e42fd549f79be76b9b2edb1745
I20260808 11:45:15.525164 29419 ts_tablet_manager.cc:1939] T f98e05e42fd549f79be76b9b2edb1745 P c506a6aa75ae4c21a8a9fdac3378bd24: Deleting consensus metadata
I20260808 11:45:15.527137 28703 catalog_manager.cc:4841] TS c506a6aa75ae4c21a8a9fdac3378bd24 (127.13.158.67:37583): tablet f98e05e42fd549f79be76b9b2edb1745 (table test-table [id=fec66f10ad3e424bb0944f023ed043c4]) successfully deleted
I20260808 11:45:15.529507 29422 tablet_replica.cc:330] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: stopping tablet replica
I20260808 11:45:15.529834 29423 tablet_replica.cc:330] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: stopping tablet replica
I20260808 11:45:15.530185 29422 raft_consensus.cc:2229] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:45:15.530265 29423 raft_consensus.cc:2229] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:45:15.530838 29422 raft_consensus.cc:2258] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:45:15.530970 29423 raft_consensus.cc:2258] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:45:15.533167 29422 ts_tablet_manager.cc:1905] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:45:15.533286 29423 ts_tablet_manager.cc:1905] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 11:45:15 dist-test-slave-39l8 krb5kdc[28663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:45:15.540427 29422 ts_tablet_manager.cc:1918] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:45:15.540593 29422 log.cc:1196] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/f98e05e42fd549f79be76b9b2edb1745
I20260808 11:45:15.540702 29423 ts_tablet_manager.cc:1918] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:45:15.541038 29423 log.cc:1196] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/f98e05e42fd549f79be76b9b2edb1745
I20260808 11:45:15.541288 29422 ts_tablet_manager.cc:1939] T f98e05e42fd549f79be76b9b2edb1745 P f9ad408e40ad4f9aa17dfe024159dc71: Deleting consensus metadata
I20260808 11:45:15.541695 29423 ts_tablet_manager.cc:1939] T f98e05e42fd549f79be76b9b2edb1745 P e194b41465724527ac498242f9acde95: Deleting consensus metadata
I20260808 11:45:15.543537 28702 catalog_manager.cc:4841] TS e194b41465724527ac498242f9acde95 (127.13.158.65:35287): tablet f98e05e42fd549f79be76b9b2edb1745 (table test-table [id=fec66f10ad3e424bb0944f023ed043c4]) successfully deleted
I20260808 11:45:15.543880 28702 catalog_manager.cc:4841] TS f9ad408e40ad4f9aa17dfe024159dc71 (127.13.158.66:33259): tablet f98e05e42fd549f79be76b9b2edb1745 (table test-table [id=fec66f10ad3e424bb0944f023ed043c4]) successfully deleted
I20260808 11:45:15.546085 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.500404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.13.158.65:35287)
0808 11:45:15.500558 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.500566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.525885 (+ 25319us) server_negotiation.cc:407] Connection header received
0808 11:45:15.526462 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.526469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.526555 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.526635 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:15.528279 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.529662 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.531473 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.532359 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.532511 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.537783 (+ 5272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:15.537797 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:15.537816 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:15.537870 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:15.542616 (+ 4746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.543665 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.543673 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.543677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.543848 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.544328 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.544333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.544336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.544811 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:15.545108 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.545414 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.545718 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 11:45:15.546947 28962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:15.644537 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 28802
I20260808 11:45:15.668418 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 28982
Aug 08 11:45:15 dist-test-slave-39l8 krb5kdc[28663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:45:15.682121 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.669131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.13.158.66:33259)
0808 11:45:15.669302 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.669306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.669361 (+ 55us) server_negotiation.cc:407] Connection header received
0808 11:45:15.669519 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.669523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.669584 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.669688 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:15.670948 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.671691 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.672836 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.673217 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.673367 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.677265 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:15.677276 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:15.677291 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:15.677328 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:15.679912 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.680481 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.680488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.680490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.680612 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.680915 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.680918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.680920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.681275 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:15.681480 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.681732 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.681881 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:45:15.682840 29153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:15.769999 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 28982
I20260808 11:45:15.796057 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 29170
Aug 08 11:45:15 dist-test-slave-39l8 krb5kdc[28663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:45:15.810132 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:45:15.796801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.13.158.67:37583)
0808 11:45:15.796969 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:45:15.796974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:45:15.796992 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:45:15.797163 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:45:15.797167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:45:15.797241 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:45:15.797346 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:45:15.798586 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.799503 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.800640 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:45:15.800997 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:45:15.801095 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:45:15.805213 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:45:15.805228 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:45:15.805245 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:45:15.805275 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:45:15.807957 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.808500 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.808507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.808509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.808647 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:45:15.808959 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:45:15.808962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:45:15.808964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:45:15.809297 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:45:15.809513 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 11:45:15.809744 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 11:45:15.809911 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:45:15.810781 29343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:15.897063 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 29170
I20260808 11:45:15.921590 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 28679
I20260808 11:45:15.922760 28756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:45:16.034752 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 28679
2026-08-08T11:45:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4416 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:45:16 dist-test-slave-39l8 krb5kdc[29440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:45:16 dist-test-slave-39l8 krb5kdc[29440](info): set up 2 sockets
Aug 08 11:45:16 dist-test-slave-39l8 krb5kdc[29440](info): commencing operation
krb5kdc: starting...
W20260808 11:45:17.313894 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:45:17 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:45:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/rangeradmin_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/rangeradmin_127.13.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/rangerlookup_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/rangerlookup_127.13.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/HTTP_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/HTTP_127.13.158.84@KRBTEST.COM.keytab.
I20260808 11:45:17.663769 13945 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskWrISxk/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/postgres -l logfile start
2026-08-08 11:45:19.956 UTC [29481] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 11:45:19.956 UTC [29481] LOG: listening on IPv4 address "127.13.158.84", port 48577
2026-08-08 11:45:19.960 UTC [29481] LOG: listening on Unix socket "/tmp/.s.PGSQL.48577"
2026-08-08 11:45:19.983 UTC [29485] LOG: database system was shut down at 2026-08-08 11:45:18 UTC
2026-08-08 11:45:19.986 UTC [29481] LOG: database system is ready to accept connections
I20260808 11:45:21.121084 13945 mini_postgres.cc:96] Postgres bound to 48577
2026-08-08 11:45:21.143 UTC [29495] FATAL: database "slave" does not exist
127.13.158.84:48577 - accepting connections
I20260808 11:45:21.144541 13945 mini_ranger.cc:161] Starting Ranger...
I20260808 11:45:21.172938 13945 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 11:45:21.615037 13945 mini_ranger.cc:87] Created ranger Postgres database
I20260808 11:45:21.615156 13945 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-admin
2026-08-08 11:45:21,933 [I] DB FLAVOR :POSTGRES
2026-08-08 11:45:21,933 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 11:45:21,933 [I] Checking connection..
2026-08-08 11:45:21,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 11:45:22,201 [I] Checking connection passed.
2026-08-08 11:45:22,201 [I] --------- Verifying version history table ---------
2026-08-08 11:45:22,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:45:22,465 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 11:45:22,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:45:22,729 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 11:45:22,729 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 11:45:22,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 11:45:22.985 UTC [29590] WARNING: there is no transaction in progress
2026-08-08 11:45:22,994 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 11:45:22,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:45:23,252 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 11:45:23,253 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 11:45:23,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 11:45:23,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 11:45:23,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-39l8.c.gcp-upstream.internal','N') ;"
2026-08-08 11:45:24,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 11:45:24,284 [I] Table x_portal_user does not exist in database ranger
2026-08-08 11:45:24,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 11:45:24,526 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 11:45:24,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 11:45:24,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 11:45:25,058 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 11:45:25,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 11:45:25.964 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:25.980 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:25.996 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.012 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.033 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.051 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.104 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.111 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.124 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.133 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.143 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.156 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.166 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.172 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.178 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26.184 UTC [29754] WARNING: there is no transaction in progress
2026-08-08 11:45:26,752 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 11:45:26,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 11:45:27,009 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 11:45:27,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 11:45:27,273 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 11:45:27,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 11:45:27,517 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 11:45:27,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 11:45:27,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 11:45:28,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-39l8.c.gcp-upstream.internal';"
2026-08-08 11:45:28,276 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 11:45:28,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 11:45:28,529 [I] DB_PATCHES have already been applied
I20260808 11:45:28.535699 13945 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-admin:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskWrISxk/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 11:45:28.535792 13945 mini_ranger.cc:193] Using host: 127.13.158.84
I20260808 11:45:28.556305 13945 mini_ranger.cc:227] Ranger admin URL: http://127.13.158.84:52135
Aug 08, 2026 11:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:45:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:45:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:45:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 11:45:29.913794 13945 mini_ranger.cc:160] Time spent starting Ranger: real 8.769s user 0.000s sys 0.056s
Aug 08, 2026 11:45:49 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:45:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 11:45:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:45:49 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:45:50 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 11:45:50.605229 13945 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/rangerkms_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/rangerkms_127.13.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.158.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.158.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.158.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/HTTP_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.158.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/HTTP_127.13.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 11:45:50 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189550, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 11:45:50.798532 13945 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 11:45:50.823662 13945 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 11:45:51.575390 13945 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 11:45:51.575564 13945 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms
I20260808 11:45:51.575704 13945 mini_ranger_kms.cc:227] Using postgres at 127.13.158.84:48577
2026-08-08 11:45:52,024 [I] DB FLAVOR :POSTGRES
2026-08-08 11:45:52,025 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 11:45:52,025 [I] Checking connection
2026-08-08 11:45:52,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 11:45:52,269 [I] connection success
2026-08-08 11:45:52,269 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 11:45:52,270 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 11:45:52,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 11:45:52,511 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 11:45:52,512 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 11:45:52,512 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 11:45:52,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:48577/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 11:45:52,762 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 11:45:53.200675 13945 mini_ranger_kms.cc:310] Created kms service
I20260808 11:45:53.367974 13945 mini_ranger_kms.cc:326] Created kudu user
I20260808 11:45:53.411451 13945 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 11:45:53 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189550, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:45:53.795539 13945 mini_ranger_kms.cc:384] Added ranger policy
I20260808 11:45:53.795936 13945 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms:/tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 11:45:53.796003 13945 mini_ranger_kms.cc:240] Using host: 127.13.158.84
I20260808 11:45:53.819941 13945 mini_ranger_kms.cc:276] Ranger KMS PID: 30028
I20260808 11:45:53.820173 13945 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.158.84:55919
Aug 08, 2026 11:45:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:45:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:45:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:45:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:45:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:45:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 11:45:56 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.158.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:45:56.577205 13945 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.779s user 0.087s sys 0.390s
I20260808 11:45:56.577481 13945 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 11:45:56.577713 13945 mini_ranger_kms.cc:401] 127.13.158.84:55919/kms/v1/keys
Aug 08 11:45:56 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.158.84@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
Aug 08 11:45:57 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.158.84@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
Aug 08 11:45:59 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:45:59.430297 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:39147
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:39147
--ranger_config_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:55919/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:45:59.864161 30096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:45:59.864506 30096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:45:59.864728 30096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:45:59.875842 30096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:45:59.876097 30096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:45:59.876267 30096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:45:59.876461 30096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:45:59.876612 30096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:45:59.891714 30096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.158.84:55919/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:39147
--ranger_config_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:39147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:45:59.893445 30096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:45:59.895119 30096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:45:59.917507 30102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:59.918498 30105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:45:59.919360 30101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:45:59.919759 30096 server_base.cc:1034] running on GCE node
I20260808 11:45:59.920652 30096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:45:59.922508 30096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:45:59.923790 30096 hybrid_clock.cc:648] HybridClock initialized: now 1786189559923684 us; error 49 us; skew 500 ppm
I20260808 11:45:59.924338 30096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:45:59 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189559, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:45:59.929533 30096 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:45:59.932284 30096 webserver.cc:460] Webserver started at http://127.13.158.126:41693/ using document root <none> and password file <none>
I20260808 11:45:59.933110 30096 fs_manager.cc:359] Metadata directory not provided
I20260808 11:45:59.933314 30096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:45:59.933771 30096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:45:59 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189559, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:00.005411 30096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1"
format_stamp: "Formatted at 2026-08-08 11:45:59 on dist-test-slave-39l8"
server_key: "5364505638d4abc89e8dabf18c6820c6"
server_key_iv: "d266867d96368bf08e272069235075e9"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:00.006592 30096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1"
format_stamp: "Formatted at 2026-08-08 11:45:59 on dist-test-slave-39l8"
server_key: "5364505638d4abc89e8dabf18c6820c6"
server_key_iv: "d266867d96368bf08e272069235075e9"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:00.015266 30096 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260808 11:46:00.039790 30115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:00.041906 30096 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260808 11:46:00.042224 30096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1"
format_stamp: "Formatted at 2026-08-08 11:45:59 on dist-test-slave-39l8"
server_key: "5364505638d4abc89e8dabf18c6820c6"
server_key_iv: "d266867d96368bf08e272069235075e9"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:00.042589 30096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:00.078868 30096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:00.084667 30096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:00.085155 30096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:00.138000 30096 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:39147
I20260808 11:46:00.138195 30190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:39147 every 8 connection(s)
I20260808 11:46:00.139916 30096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:46:00.145351 30193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:46:00.147648 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 30096
I20260808 11:46:00.147877 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:46:00.148689 13945 external_mini_cluster.cc:1428] Setting key 794e7a7c12fe81e2b4a781dba6420aec
I20260808 11:46:00.161595 30193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1: Bootstrap starting.
I20260808 11:46:00.166288 30193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1: Neither blocks nor log segments found. Creating new log.
Aug 08 11:46:00 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:00.168702 30193 log.cc:824] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:00.174167 30193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1: No bootstrap required, opened a new log
I20260808 11:46:00.180404 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:00.151486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.13.158.126:39147)
0808 11:46:00.153335 (+ 1849us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:00.153347 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:00.153409 (+ 62us) server_negotiation.cc:407] Connection header received
0808 11:46:00.156697 (+ 3288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:00.156722 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:00.157133 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:00.157575 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:00.159177 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:00.161076 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:00.162590 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:00.163520 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:00.163747 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:00.168520 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:00.168535 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:00.168556 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:00.168648 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:00.174395 (+ 5747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:00.175215 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:00.175223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:00.175234 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:00.175566 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:00.175974 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:00.175980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:00.175984 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:00.176955 (+ 971us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:00.177450 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:00.177887 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 11:46:00.178463 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1538,"thread_start_us":1178,"threads_started":1}
I20260808 11:46:00.187690 30193 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39147 } }
I20260808 11:46:00.188218 30193 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:46:00.188373 30193 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8ee51fe7a4453b88ddd24bf6df97e1, State: Initialized, Role: FOLLOWER
I20260808 11:46:00.189067 30193 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39147 } }
I20260808 11:46:00.189385 30193 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:46:00.189625 30193 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:46:00.189839 30193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:46:00.193277 30193 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39147 } }
I20260808 11:46:00.193869 30193 leader_election.cc:302] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef8ee51fe7a4453b88ddd24bf6df97e1; no voters:
I20260808 11:46:00.195174 30193 leader_election.cc:288] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:46:00.195415 30201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:46:00.196954 30201 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [term 1 LEADER]: Becoming Leader. State: Replica: ef8ee51fe7a4453b88ddd24bf6df97e1, State: Running, Role: LEADER
I20260808 11:46:00.197754 30201 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39147 } }
I20260808 11:46:00.198305 30193 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:46:00.202127 30204 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39147 } } }
I20260808 11:46:00.203289 30204 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [sys.catalog]: This master's current role is: LEADER
I20260808 11:46:00.203017 30206 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef8ee51fe7a4453b88ddd24bf6df97e1. Latest consensus state: current_term: 1 leader_uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8ee51fe7a4453b88ddd24bf6df97e1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39147 } } }
I20260808 11:46:00.204026 30206 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1 [sys.catalog]: This master's current role is: LEADER
I20260808 11:46:00.215921 30215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:46:00.224155 30215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:46:00.225314 30193 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 11:46:00.252743 30215 catalog_manager.cc:1347] Generated new cluster ID: cc0e49af0ffd49c683f26f856e1840c1
I20260808 11:46:00.252916 30215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:46:00.282819 30215 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:46:00.285599 30215 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:46:00.304589 30215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1: Generated new TSK 0
I20260808 11:46:00.305981 30215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:46:01.232610 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:55919/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39147
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:46:01 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189561, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 11:46:01.780040 30259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:01.780567 30259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:01.780779 30259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:01.792845 30259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:01.793012 30259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:01.793267 30259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:46:01.809877 30259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.158.84:55919/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:01.811825 30259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:01.813911 30259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:01.837879 30276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:01.838222 30277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:01.846892 30280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:01.848564 30259 server_base.cc:1034] running on GCE node
I20260808 11:46:01.849931 30259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:01.852213 30259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:01.853554 30259 hybrid_clock.cc:648] HybridClock initialized: now 1786189561853403 us; error 117 us; skew 500 ppm
I20260808 11:46:01.854048 30259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:01 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189561, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:01.860854 30259 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:46:01.869263 30259 webserver.cc:460] Webserver started at http://127.13.158.65:33387/ using document root <none> and password file <none>
I20260808 11:46:01.870303 30259 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:01.870563 30259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:46:01.871157 30259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:46:01 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189561, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:01.895066 30259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "013fb92c6a49416ca892ff4fb7c45a6f"
format_stamp: "Formatted at 2026-08-08 11:46:01 on dist-test-slave-39l8"
server_key: "03e128dad678cb08c31faad49018d77c"
server_key_iv: "f36b9512476457dfb94be8f33d8442a6"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:01.897094 30259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "013fb92c6a49416ca892ff4fb7c45a6f"
format_stamp: "Formatted at 2026-08-08 11:46:01 on dist-test-slave-39l8"
server_key: "03e128dad678cb08c31faad49018d77c"
server_key_iv: "f36b9512476457dfb94be8f33d8442a6"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:01.910297 30259 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.007s
I20260808 11:46:01.938104 30292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:01.940308 30259 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260808 11:46:01.940691 30259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "013fb92c6a49416ca892ff4fb7c45a6f"
format_stamp: "Formatted at 2026-08-08 11:46:01 on dist-test-slave-39l8"
server_key: "03e128dad678cb08c31faad49018d77c"
server_key_iv: "f36b9512476457dfb94be8f33d8442a6"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:01.941023 30259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:01.966310 30259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:01.974107 30259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:01.974610 30259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:01.976564 30259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:01.979095 30259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:01.979334 30259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:01.979571 30259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:01.979707 30259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:02.197279 30259 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:37483
I20260808 11:46:02.198836 30471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:37483 every 8 connection(s)
I20260808 11:46:02.199641 30259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:46:02.207806 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 30259
I20260808 11:46:02.208088 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:46:02.209087 13945 external_mini_cluster.cc:1428] Setting key 29cb02f0fc52e122e93580feba32fd56
Aug 08 11:46:02 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189561, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
Aug 08 11:46:02 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189561, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
I20260808 11:46:02.262006 30476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:02.208099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39147 (local address 127.13.158.65:36939)
0808 11:46:02.223314 (+ 15215us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:02.223384 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:02.229455 (+ 6071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:02.237542 (+ 8087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:02.237559 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:02.238643 (+ 1084us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:02.239906 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:02.239940 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:02.241772 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:02.241778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:02.243855 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:02.243882 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:02.245394 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:02.245404 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:02.245879 (+ 475us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:02.246967 (+ 1088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:02.247007 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:02.251417 (+ 4410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:02.256455 (+ 5038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:02.256466 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:02.256487 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:02.257143 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:02.257671 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:02.257676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:02.257679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:02.258022 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:02.258809 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:02.258818 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:02.259371 (+ 553us) client_negotiation.cc:769] Sending connection context
0808 11:46:02.259735 (+ 364us) client_negotiation.cc:240] Negotiation successful
0808 11:46:02.260348 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14725,"thread_start_us":8968,"threads_started":1}
I20260808 11:46:02.263363 30477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:02.214965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:36939 (local address 127.13.158.126:39147)
0808 11:46:02.227433 (+ 12468us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:02.227439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:02.227468 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:46:02.234781 (+ 7313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:02.234791 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:02.234971 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:02.236821 (+ 1850us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:02.240255 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:02.241399 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:02.244066 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:02.244716 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:02.244873 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:02.251909 (+ 7036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:02.251917 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:02.251923 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:02.251969 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:02.256249 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:02.257329 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:02.257336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:02.257339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:02.257526 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:02.258168 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:02.258174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:02.258177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:02.258602 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:02.262479 (+ 3877us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:02.262648 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 11:46:02.262930 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12290,"thread_start_us":12358,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
I20260808 11:46:02.266110 30472 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39147
I20260808 11:46:02.266582 30472 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:02.268018 30472 heartbeater.cc:511] Master 127.13.158.126:39147 requested a full tablet report, sending...
I20260808 11:46:02.277446 30135 ts_manager.cc:194] Registered new tserver with Master: 013fb92c6a49416ca892ff4fb7c45a6f (127.13.158.65:37483)
I20260808 11:46:02.280380 30135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:36939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
W20260808 11:46:02.319366 13945 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.13.158.66: real 0.107s user 0.000s sys 0.027s
I20260808 11:46:02.320425 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:55919/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39147
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:46:02 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189561, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
W20260808 11:46:03.027613 30480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:03.028080 30480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:03.028362 30480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:03.037879 30480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:03.038112 30480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:03.038384 30480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:46:03.052199 30480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.158.84:55919/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:03.053761 30480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:03.055315 30480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:03.077484 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:03.077530 30506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:03.078895 30501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:03.080628 30480 server_base.cc:1034] running on GCE node
I20260808 11:46:03.081560 30480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:03.082676 30480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:03.084018 30480 hybrid_clock.cc:648] HybridClock initialized: now 1786189563083907 us; error 107 us; skew 500 ppm
I20260808 11:46:03.084371 30480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:03 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189563, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:03.093619 30480 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:46:03.096098 30480 webserver.cc:460] Webserver started at http://127.13.158.66:38315/ using document root <none> and password file <none>
I20260808 11:46:03.097309 30480 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:03.097522 30480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:46:03.097983 30480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:46:03 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189563, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:03.120033 30480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "3beb402360c147b28840e08511768464"
format_stamp: "Formatted at 2026-08-08 11:46:03 on dist-test-slave-39l8"
server_key: "67c2ccc1d1b742a4f21c09feb13bc1ff"
server_key_iv: "f035d0a32f3d031081ae14b30a9615d7"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:03.121048 30480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "3beb402360c147b28840e08511768464"
format_stamp: "Formatted at 2026-08-08 11:46:03 on dist-test-slave-39l8"
server_key: "67c2ccc1d1b742a4f21c09feb13bc1ff"
server_key_iv: "f035d0a32f3d031081ae14b30a9615d7"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:03.127792 30480 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260808 11:46:03.160261 30517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:03.162528 30480 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.005s sys 0.005s
I20260808 11:46:03.162868 30480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "3beb402360c147b28840e08511768464"
format_stamp: "Formatted at 2026-08-08 11:46:03 on dist-test-slave-39l8"
server_key: "67c2ccc1d1b742a4f21c09feb13bc1ff"
server_key_iv: "f035d0a32f3d031081ae14b30a9615d7"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:03.163259 30480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:03.215093 30480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:03.220746 30480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:03.221243 30480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:03.222909 30480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:03.225062 30480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:03.225194 30480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:03.225426 30480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:03.225541 30480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:03.284909 30472 heartbeater.cc:503] Master 127.13.158.126:39147 was elected leader, sending a full tablet report...
I20260808 11:46:03.339463 30480 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:41013
I20260808 11:46:03.339690 30686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:41013 every 8 connection(s)
I20260808 11:46:03.340992 30480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:46:03.348057 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 30480
I20260808 11:46:03.348224 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:46:03.348841 13945 external_mini_cluster.cc:1428] Setting key 4de8e6ebfb9d688ed83623d49b11ebd5
Aug 08 11:46:03 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189563, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:03.375480 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:03.344693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:34013 (local address 127.13.158.126:39147)
0808 11:46:03.345480 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:03.345487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:03.349277 (+ 3790us) server_negotiation.cc:407] Connection header received
0808 11:46:03.353009 (+ 3732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:03.353014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:03.353130 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:03.353237 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:03.355459 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:03.356992 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:03.359970 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:03.360842 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:03.360940 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:03.366480 (+ 5540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:03.366487 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:03.366492 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:03.366543 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:03.371027 (+ 4484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:03.372065 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:03.372070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:03.372074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:03.372260 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:03.372808 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:03.372813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:03.372816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:03.373199 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:03.373343 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:03.374461 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0808 11:46:03.374826 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":405,"threads_started":1}
I20260808 11:46:03.376293 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:03.346060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39147 (local address 127.13.158.66:34013)
0808 11:46:03.347325 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:03.347363 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:03.352599 (+ 5236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:03.353730 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:03.353742 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:03.354394 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:03.355235 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:03.355260 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:03.357275 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:03.357281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:03.359762 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:03.359818 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:03.361439 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:03.361448 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:03.361762 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:03.362850 (+ 1088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:03.362902 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:03.366193 (+ 3291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:03.371252 (+ 5059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:03.371261 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:03.371280 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:03.371893 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:03.372417 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:03.372422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:03.372425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:03.372678 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:03.373436 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:03.373444 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:03.373938 (+ 494us) client_negotiation.cc:769] Sending connection context
0808 11:46:03.374263 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 11:46:03.374910 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"thread_start_us":778,"threads_started":1}
I20260808 11:46:03.378299 30687 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39147
I20260808 11:46:03.378803 30687 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:03.380389 30687 heartbeater.cc:511] Master 127.13.158.126:39147 requested a full tablet report, sending...
I20260808 11:46:03.382470 30135 ts_manager.cc:194] Registered new tserver with Master: 3beb402360c147b28840e08511768464 (127.13.158.66:41013)
I20260808 11:46:03.384049 30135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:34013
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:46:03.426668 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:55919/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39147
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:46:03.981933 30696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:03.982337 30696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:03.982609 30696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:03.992034 30696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:03.992268 30696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:03.992494 30696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:46:04.004851 30696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.158.84:55919/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:04.006573 30696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:04.008144 30696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:04.028649 30704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:04.028478 30705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:04.028877 30707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:04.030277 30696 server_base.cc:1034] running on GCE node
I20260808 11:46:04.031190 30696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:04.032305 30696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:04.033545 30696 hybrid_clock.cc:648] HybridClock initialized: now 1786189564033470 us; error 49 us; skew 500 ppm
I20260808 11:46:04.033983 30696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:04 dist-test-slave-39l8 krb5kdc[29440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189564, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:04.038461 30696 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:46:04.040150 30696 webserver.cc:460] Webserver started at http://127.13.158.67:45929/ using document root <none> and password file <none>
I20260808 11:46:04.040997 30696 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:04.041180 30696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:46:04.041571 30696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:46:04 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189564, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:04.059789 30696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "bfa710344b07425995a4ff05518b7035"
format_stamp: "Formatted at 2026-08-08 11:46:04 on dist-test-slave-39l8"
server_key: "d9e13f0637c19d7a30445e4bea252ac3"
server_key_iv: "0035c6838fd29121398f9aff09624038"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:04.060829 30696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "bfa710344b07425995a4ff05518b7035"
format_stamp: "Formatted at 2026-08-08 11:46:04 on dist-test-slave-39l8"
server_key: "d9e13f0637c19d7a30445e4bea252ac3"
server_key_iv: "0035c6838fd29121398f9aff09624038"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:04.068101 30696 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.006s
I20260808 11:46:04.086555 30716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:04.088268 30696 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260808 11:46:04.088557 30696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "bfa710344b07425995a4ff05518b7035"
format_stamp: "Formatted at 2026-08-08 11:46:04 on dist-test-slave-39l8"
server_key: "d9e13f0637c19d7a30445e4bea252ac3"
server_key_iv: "0035c6838fd29121398f9aff09624038"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:04.088883 30696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:04.106396 30696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:04.112398 30696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:04.112946 30696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:04.114493 30696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:04.116763 30696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:04.116927 30696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:04.117115 30696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:04.117240 30696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:04.204039 30696 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:37387
I20260808 11:46:04.204190 30881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:37387 every 8 connection(s)
I20260808 11:46:04.205518 30696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:46:04.210280 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 30696
I20260808 11:46:04.210482 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:46:04.211083 13945 external_mini_cluster.cc:1428] Setting key f3cb152c1debb7501a6e7461c00f00e9
Aug 08 11:46:04 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189564, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:04.232331 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.209073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:54243 (local address 127.13.158.126:39147)
0808 11:46:04.209916 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.209923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.211466 (+ 1543us) server_negotiation.cc:407] Connection header received
0808 11:46:04.214610 (+ 3144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.214614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.214785 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.214874 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:04.216677 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.217731 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.220042 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.220653 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.220759 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.224811 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:04.224818 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:04.224823 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:04.224894 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:04.228891 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.229736 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.229740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.229742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.229874 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.230322 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.230326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.230328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.230774 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:04.230889 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.231765 (+ 876us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.231957 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":519,"threads_started":1}
I20260808 11:46:04.233522 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.209440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39147 (local address 127.13.158.67:54243)
0808 11:46:04.211252 (+ 1812us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:04.211301 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:04.214333 (+ 3032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:04.215130 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:04.215139 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:04.215729 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:04.216532 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.216552 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.217894 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.217898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.219858 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.219918 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.220887 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.220892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.221169 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.221897 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:04.221918 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:04.224497 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:04.229012 (+ 4515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:04.229037 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:04.229048 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:04.229612 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:04.229967 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:04.229970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:04.229972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:04.230218 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:04.230920 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:04.230926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:04.231261 (+ 335us) client_negotiation.cc:769] Sending connection context
0808 11:46:04.231681 (+ 420us) client_negotiation.cc:240] Negotiation successful
0808 11:46:04.232185 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1442,"thread_start_us":1085,"threads_started":1}
I20260808 11:46:04.235095 30883 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39147
I20260808 11:46:04.235430 30883 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:04.236311 30883 heartbeater.cc:511] Master 127.13.158.126:39147 requested a full tablet report, sending...
I20260808 11:46:04.238016 30135 ts_manager.cc:194] Registered new tserver with Master: bfa710344b07425995a4ff05518b7035 (127.13.158.67:37387)
I20260808 11:46:04.239377 30135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:54243
I20260808 11:46:04.239816 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:46:04.262300 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.245727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.13.158.126:39147)
0808 11:46:04.245880 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.245884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.246971 (+ 1087us) server_negotiation.cc:407] Connection header received
0808 11:46:04.247264 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.247268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.247380 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.247461 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:04.249563 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.250678 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.252501 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.252975 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.253067 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.255998 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:04.256003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:04.256006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:04.256035 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:04.259476 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.260303 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.260306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.260309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.260487 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.260911 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.260914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.260916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.261218 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:04.261485 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.261801 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.261991 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:46:04.266794 30135 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:46:04.276808 30135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:46:04.308801 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.294065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58270 (local address 127.13.158.67:37387)
0808 11:46:04.295570 (+ 1505us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.295580 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.297532 (+ 1952us) server_negotiation.cc:407] Connection header received
0808 11:46:04.297918 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.297924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.298177 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.298378 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:04.300237 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.302435 (+ 2198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.305539 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.307064 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.307237 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.307700 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.308059 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.308293 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1229,"thread_start_us":1011,"threads_started":1}
I20260808 11:46:04.308815 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.293494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37387 (local address 127.0.0.1:58270)
0808 11:46:04.297380 (+ 3886us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:04.297431 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:04.297683 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:04.298386 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:04.298393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:04.298425 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:04.299620 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.299667 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.302734 (+ 3067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.302746 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.305362 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.305375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.307442 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.307464 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.307728 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.307805 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 11:46:04.307952 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 11:46:04.308218 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3516,"thread_start_us":3241,"threads_started":1}
I20260808 11:46:04.314069 30780 tablet_service.cc:1467] Processing CreateTablet for tablet 68b72eaf36d74b208c45b3994fb5345d (DEFAULT_TABLE table=test-table [id=a7b52ce1e29d40f0b836d8518a85d53f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:46:04.315356 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.293512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37483 (local address 127.0.0.1:52034)
0808 11:46:04.299054 (+ 5542us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:04.299073 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:04.299319 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:04.300681 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:04.300688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:04.300722 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:04.302388 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.302405 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.305558 (+ 3153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.305563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.311147 (+ 5584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.311164 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.313728 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.313737 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.314000 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.314061 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 11:46:04.314182 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 11:46:04.314328 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5321,"mutex_wait_us":52,"thread_start_us":4795,"threads_started":1}
I20260808 11:46:04.316253 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.299937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41013 (local address 127.0.0.1:35376)
0808 11:46:04.301577 (+ 1640us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:04.301595 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:04.301733 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:04.302664 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:04.302669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:04.302713 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:04.306607 (+ 3894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.306623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.309429 (+ 2806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.309434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.312304 (+ 2870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.312317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.314958 (+ 2641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.314967 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.315338 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.315391 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 11:46:04.315501 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 11:46:04.315630 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1430,"thread_start_us":1027,"threads_started":1}
I20260808 11:46:04.316664 30780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68b72eaf36d74b208c45b3994fb5345d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:46:04.316921 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.293675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.13.158.65:37483)
0808 11:46:04.295659 (+ 1984us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.295669 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.299169 (+ 3500us) server_negotiation.cc:407] Connection header received
0808 11:46:04.299556 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.299566 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.299904 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.300158 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:04.303786 (+ 3628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.305389 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.311363 (+ 5974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.313383 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.313514 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.313791 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.314282 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.314566 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1757,"thread_start_us":730,"threads_started":1}
I20260808 11:46:04.318230 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.299345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.13.158.66:41013)
0808 11:46:04.301630 (+ 2285us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.301636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.301658 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:46:04.302009 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.302018 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.302374 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.302580 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:04.307042 (+ 4462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.309220 (+ 2178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.312461 (+ 3241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.314116 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.314284 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.314499 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.316549 (+ 2050us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.316737 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2109,"thread_start_us":1941,"threads_started":1}
I20260808 11:46:04.321079 30363 tablet_service.cc:1467] Processing CreateTablet for tablet 68b72eaf36d74b208c45b3994fb5345d (DEFAULT_TABLE table=test-table [id=a7b52ce1e29d40f0b836d8518a85d53f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:46:04.322464 30363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68b72eaf36d74b208c45b3994fb5345d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:46:04.323572 30584 tablet_service.cc:1467] Processing CreateTablet for tablet 68b72eaf36d74b208c45b3994fb5345d (DEFAULT_TABLE table=test-table [id=a7b52ce1e29d40f0b836d8518a85d53f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:46:04.325306 30584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68b72eaf36d74b208c45b3994fb5345d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:46:04.336992 30912 tablet_bootstrap.cc:492] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Bootstrap starting.
I20260808 11:46:04.342356 30912 tablet_bootstrap.cc:654] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Neither blocks nor log segments found. Creating new log.
I20260808 11:46:04.346633 30912 log.cc:824] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:04.347733 30913 tablet_bootstrap.cc:492] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Bootstrap starting.
I20260808 11:46:04.354622 30913 tablet_bootstrap.cc:654] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Neither blocks nor log segments found. Creating new log.
I20260808 11:46:04.356232 30915 tablet_bootstrap.cc:492] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Bootstrap starting.
I20260808 11:46:04.356958 30913 log.cc:824] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:04.360620 30912 tablet_bootstrap.cc:492] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: No bootstrap required, opened a new log
I20260808 11:46:04.360966 30912 ts_tablet_manager.cc:1397] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.000s
I20260808 11:46:04.361778 30913 tablet_bootstrap.cc:492] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: No bootstrap required, opened a new log
I20260808 11:46:04.362196 30913 ts_tablet_manager.cc:1397] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.011s
I20260808 11:46:04.362339 30915 tablet_bootstrap.cc:654] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Neither blocks nor log segments found. Creating new log.
I20260808 11:46:04.364661 30915 log.cc:824] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:04.370507 30915 tablet_bootstrap.cc:492] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: No bootstrap required, opened a new log
I20260808 11:46:04.371011 30915 ts_tablet_manager.cc:1397] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260808 11:46:04.371433 30912 raft_consensus.cc:348] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.371940 30912 raft_consensus.cc:374] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:46:04.372128 30912 raft_consensus.cc:729] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 013fb92c6a49416ca892ff4fb7c45a6f, State: Initialized, Role: FOLLOWER
I20260808 11:46:04.372960 30912 consensus_queue.cc:260] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.376268 30912 ts_tablet_manager.cc:1428] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Time spent starting tablet: real 0.015s user 0.007s sys 0.009s
I20260808 11:46:04.377264 30913 raft_consensus.cc:348] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.377745 30913 raft_consensus.cc:374] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:46:04.377859 30913 raft_consensus.cc:729] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa710344b07425995a4ff05518b7035, State: Initialized, Role: FOLLOWER
I20260808 11:46:04.378628 30913 consensus_queue.cc:260] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.381441 30883 heartbeater.cc:503] Master 127.13.158.126:39147 was elected leader, sending a full tablet report...
I20260808 11:46:04.382015 30913 ts_tablet_manager.cc:1428] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Time spent starting tablet: real 0.020s user 0.014s sys 0.008s
I20260808 11:46:04.384550 30915 raft_consensus.cc:348] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.385133 30915 raft_consensus.cc:374] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:46:04.385346 30915 raft_consensus.cc:729] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3beb402360c147b28840e08511768464, State: Initialized, Role: FOLLOWER
I20260808 11:46:04.386150 30915 consensus_queue.cc:260] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.387494 30687 heartbeater.cc:503] Master 127.13.158.126:39147 was elected leader, sending a full tablet report...
I20260808 11:46:04.387862 30915 ts_tablet_manager.cc:1428] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
W20260808 11:46:04.457808 30475 tablet.cc:2369] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:46:04.458276 30884 tablet.cc:2369] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:46:04.523670 30921 raft_consensus.cc:482] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:46:04.524101 30921 raft_consensus.cc:504] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.526341 30921 leader_election.cc:288] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa710344b07425995a4ff05518b7035 (127.13.158.67:37387), 3beb402360c147b28840e08511768464 (127.13.158.66:41013)
I20260808 11:46:04.541901 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.527214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:51121 (local address 127.13.158.67:37387)
0808 11:46:04.527476 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.527483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.530486 (+ 3003us) server_negotiation.cc:407] Connection header received
0808 11:46:04.530775 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.530797 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.530873 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.530945 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:04.532609 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.534256 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.538234 (+ 3978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.540193 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.540357 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.540435 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.541370 (+ 935us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.541516 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:46:04.541940 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.526933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:52601 (local address 127.13.158.66:41013)
0808 11:46:04.527302 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.527321 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.530328 (+ 3007us) server_negotiation.cc:407] Connection header received
0808 11:46:04.530734 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.530741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.530876 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.530963 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:04.532630 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.534229 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.538224 (+ 3995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.540145 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.540361 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.540435 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.541371 (+ 936us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.541527 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 11:46:04.542436 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.528037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37387 (local address 127.13.158.65:51121)
0808 11:46:04.530379 (+ 2342us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:04.530395 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:04.530563 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:04.531221 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:04.531227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:04.531255 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:04.532250 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.532266 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.534419 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.534424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.538079 (+ 3655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.538088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.540823 (+ 2735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.540830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.541109 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.541219 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 11:46:04.541326 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 11:46:04.541539 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2132,"mutex_wait_us":155,"thread_start_us":1413,"threads_started":1}
I20260808 11:46:04.542658 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.527959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41013 (local address 127.13.158.65:52601)
0808 11:46:04.530176 (+ 2217us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:04.530193 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:04.530506 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:04.531222 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:04.531227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:04.531370 (+ 143us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:04.532473 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.532490 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.534459 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.534466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.538039 (+ 3573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:04.538069 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.540685 (+ 2616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.540694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:04.541106 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.541219 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 11:46:04.541326 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 11:46:04.541538 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2066,"mutex_wait_us":139,"spinlock_wait_cycles":23936,"thread_start_us":1503,"threads_started":1}
I20260808 11:46:04.543841 30810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68b72eaf36d74b208c45b3994fb5345d" candidate_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa710344b07425995a4ff05518b7035" is_pre_election: true
I20260808 11:46:04.543895 30620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68b72eaf36d74b208c45b3994fb5345d" candidate_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3beb402360c147b28840e08511768464" is_pre_election: true
I20260808 11:46:04.544485 30810 raft_consensus.cc:2454] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 013fb92c6a49416ca892ff4fb7c45a6f in term 0.
I20260808 11:46:04.544706 30620 raft_consensus.cc:2454] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 013fb92c6a49416ca892ff4fb7c45a6f in term 0.
I20260808 11:46:04.545405 30294 leader_election.cc:302] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 013fb92c6a49416ca892ff4fb7c45a6f, bfa710344b07425995a4ff05518b7035; no voters:
I20260808 11:46:04.546083 30921 raft_consensus.cc:2781] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:46:04.546352 30921 raft_consensus.cc:482] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:46:04.546547 30921 raft_consensus.cc:3037] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:46:04.549793 30921 raft_consensus.cc:504] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.550554 30921 leader_election.cc:288] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [CANDIDATE]: Term 1 election: Requested vote from peers bfa710344b07425995a4ff05518b7035 (127.13.158.67:37387), 3beb402360c147b28840e08511768464 (127.13.158.66:41013)
I20260808 11:46:04.551213 30620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68b72eaf36d74b208c45b3994fb5345d" candidate_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3beb402360c147b28840e08511768464"
I20260808 11:46:04.551291 30810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68b72eaf36d74b208c45b3994fb5345d" candidate_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa710344b07425995a4ff05518b7035"
I20260808 11:46:04.551510 30620 raft_consensus.cc:3037] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:46:04.551549 30810 raft_consensus.cc:3037] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:46:04.555982 30620 raft_consensus.cc:2454] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 013fb92c6a49416ca892ff4fb7c45a6f in term 1.
I20260808 11:46:04.555982 30810 raft_consensus.cc:2454] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 013fb92c6a49416ca892ff4fb7c45a6f in term 1.
I20260808 11:46:04.556552 30298 leader_election.cc:302] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 013fb92c6a49416ca892ff4fb7c45a6f, 3beb402360c147b28840e08511768464; no voters:
I20260808 11:46:04.556979 30921 raft_consensus.cc:2781] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:46:04.557716 30921 raft_consensus.cc:686] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 1 LEADER]: Becoming Leader. State: Replica: 013fb92c6a49416ca892ff4fb7c45a6f, State: Running, Role: LEADER
I20260808 11:46:04.558349 30921 consensus_queue.cc:237] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } }
I20260808 11:46:04.563931 30135 catalog_manager.cc:5495] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 013fb92c6a49416ca892ff4fb7c45a6f (127.13.158.65). New cstate: current_term: 1 leader_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "013fb92c6a49416ca892ff4fb7c45a6f" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 } health_report { overall_health: UNKNOWN } } }
W20260808 11:46:04.594514 30689 tablet.cc:2369] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:46:04.603106 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.596284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52050 (local address 127.13.158.65:37483)
0808 11:46:04.596694 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.596703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.596725 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:46:04.596852 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.596856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.596931 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.597088 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:46:04.597559 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.598654 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.600623 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.601214 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.601358 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.601922 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:46:04.602423 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:46:04.602585 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.602695 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.602796 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260808 11:46:04.613052 30620 raft_consensus.cc:1261] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 1 FOLLOWER]: Refusing update from remote peer 013fb92c6a49416ca892ff4fb7c45a6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:46:04.613072 30810 raft_consensus.cc:1261] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 1 FOLLOWER]: Refusing update from remote peer 013fb92c6a49416ca892ff4fb7c45a6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:46:04.614166 30929 consensus_queue.cc:1035] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa710344b07425995a4ff05518b7035" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:46:04.614791 30921 consensus_queue.cc:1035] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3beb402360c147b28840e08511768464" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:46:04.650379 30134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:46:04.650853 30134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:46:04.655285 30134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef8ee51fe7a4453b88ddd24bf6df97e1: Sending DeleteTablet for 3 replicas of tablet 68b72eaf36d74b208c45b3994fb5345d
I20260808 11:46:04.656610 30780 tablet_service.cc:1514] Processing DeleteTablet for tablet 68b72eaf36d74b208c45b3994fb5345d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:46:04 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:58270
I20260808 11:46:04.656780 30584 tablet_service.cc:1514] Processing DeleteTablet for tablet 68b72eaf36d74b208c45b3994fb5345d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:46:04 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:35376
I20260808 11:46:04.657130 30363 tablet_service.cc:1514] Processing DeleteTablet for tablet 68b72eaf36d74b208c45b3994fb5345d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:46:04 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:52034
I20260808 11:46:04.658495 30952 tablet_replica.cc:330] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: stopping tablet replica
I20260808 11:46:04.659039 30952 raft_consensus.cc:2229] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:46:04.659798 30952 raft_consensus.cc:2258] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:46:04.659857 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 30259
I20260808 11:46:04.664423 30952 ts_tablet_manager.cc:1905] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:46:04.667465 30955 tablet_replica.cc:330] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: stopping tablet replica
I20260808 11:46:04.668037 30955 raft_consensus.cc:2229] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:46:04.670248 30955 raft_consensus.cc:2258] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:46:04.673216 30955 ts_tablet_manager.cc:1905] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:46:04.673614 30954 tablet_replica.cc:330] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: stopping tablet replica
I20260808 11:46:04.673995 30954 raft_consensus.cc:2229] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:46:04.674491 30954 raft_consensus.cc:2258] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:46:04.674646 30952 ts_tablet_manager.cc:1918] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:46:04.674909 30952 log.cc:1196] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/68b72eaf36d74b208c45b3994fb5345d
I20260808 11:46:04.675622 30952 ts_tablet_manager.cc:1939] T 68b72eaf36d74b208c45b3994fb5345d P bfa710344b07425995a4ff05518b7035: Deleting consensus metadata
I20260808 11:46:04.677101 30954 ts_tablet_manager.cc:1905] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:46:04.677724 30119 catalog_manager.cc:4841] TS bfa710344b07425995a4ff05518b7035 (127.13.158.67:37387): tablet 68b72eaf36d74b208c45b3994fb5345d (table test-table [id=a7b52ce1e29d40f0b836d8518a85d53f]) successfully deleted
Aug 08 11:46:04 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:46:04.686420 30954 ts_tablet_manager.cc:1918] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:46:04.686733 30954 log.cc:1196] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/68b72eaf36d74b208c45b3994fb5345d
I20260808 11:46:04.686767 30955 ts_tablet_manager.cc:1918] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:46:04.687403 30955 log.cc:1196] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/68b72eaf36d74b208c45b3994fb5345d
I20260808 11:46:04.687695 30954 ts_tablet_manager.cc:1939] T 68b72eaf36d74b208c45b3994fb5345d P 3beb402360c147b28840e08511768464: Deleting consensus metadata
I20260808 11:46:04.688753 30955 ts_tablet_manager.cc:1939] T 68b72eaf36d74b208c45b3994fb5345d P 013fb92c6a49416ca892ff4fb7c45a6f: Deleting consensus metadata
I20260808 11:46:04.689839 30121 catalog_manager.cc:4841] TS 3beb402360c147b28840e08511768464 (127.13.158.66:41013): tablet 68b72eaf36d74b208c45b3994fb5345d (table test-table [id=a7b52ce1e29d40f0b836d8518a85d53f]) successfully deleted
I20260808 11:46:04.691385 30121 catalog_manager.cc:4841] TS 013fb92c6a49416ca892ff4fb7c45a6f (127.13.158.65:37483): tablet 68b72eaf36d74b208c45b3994fb5345d (table test-table [id=a7b52ce1e29d40f0b836d8518a85d53f]) successfully deleted
I20260808 11:46:04.697034 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.660817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.13.158.65:37483)
0808 11:46:04.661050 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.661057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.671479 (+ 10422us) server_negotiation.cc:407] Connection header received
0808 11:46:04.671718 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.671723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.671803 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.671911 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:04.674268 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.675594 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.678207 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.679221 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.679388 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.686573 (+ 7185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:04.686594 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:04.686624 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:04.686744 (+ 120us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:04.693066 (+ 6322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.694188 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.694199 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.694202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.694589 (+ 387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.695222 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.695226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.695228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.695642 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:04.695928 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.696367 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.696583 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:46:04.697831 30466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:04.797792 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 30259
I20260808 11:46:04.823472 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 30480
Aug 08 11:46:04 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:46:04.838063 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.824195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.13.158.66:41013)
0808 11:46:04.824350 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.824354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.824371 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:46:04.824524 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.824535 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.824612 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.824692 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:04.825925 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.826640 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.828220 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.828591 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.828713 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.832790 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:04.832804 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:04.832820 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:04.832853 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:04.835598 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.836340 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.836348 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.836350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.836484 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.836802 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.836805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.836807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.837158 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:04.837368 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.837609 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.837803 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 11:46:04.838927 30681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:04.929217 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 30480
I20260808 11:46:04.955233 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 30696
Aug 08 11:46:04 dist-test-slave-39l8 krb5kdc[29440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:46:04.969879 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:04.956057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58276 (local address 127.13.158.67:37387)
0808 11:46:04.956288 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:04.956293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:04.956311 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:46:04.956441 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:04.956446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:04.956522 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:04.956605 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:04.957881 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.958559 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.959816 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:04.960181 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:04.960269 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:04.964523 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:04.964539 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:04.964556 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:04.964586 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:04.967459 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.968058 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.968065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.968067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.968236 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:04.968538 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:04.968541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:04.968543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:04.968900 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:04.969118 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:04.969387 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 11:46:04.969531 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 11:46:04.970542 30876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:05.059656 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 30696
I20260808 11:46:05.083932 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 30096
I20260808 11:46:05.085216 30185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:05.201647 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 30096
I20260808 11:46:05.233325 13945 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 11:46:05.578444 13945 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 11:46:05.578667 13945 mini_ranger.cc:66] Stopping Ranger...
I20260808 11:46:05.923822 13945 mini_ranger.cc:68] Stopped Ranger
2026-08-08 11:46:05.924 UTC [29481] LOG: received smart shutdown request
2026-08-08 11:46:05.927 UTC [29481] LOG: background worker "logical replication launcher" (PID 29492) exited with exit code 1
2026-08-08 11:46:05.927 UTC [29486] LOG: shutting down
2026-08-08 11:46:06.078 UTC [29481] LOG: database system is shut down
2026-08-08T11:46:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50170 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:46:06 dist-test-slave-39l8 krb5kdc[30977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:46:06 dist-test-slave-39l8 krb5kdc[30977](info): set up 2 sockets
Aug 08 11:46:06 dist-test-slave-39l8 krb5kdc[30977](info): commencing operation
krb5kdc: starting...
W20260808 11:46:07.524811 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:46:07 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:46:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:46:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/rangeradmin_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/rangeradmin_127.13.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/rangerlookup_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/rangerlookup_127.13.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/HTTP_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/HTTP_127.13.158.84@KRBTEST.COM.keytab.
I20260808 11:46:07.884964 13945 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskWrISxk/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/postgres -l logfile start
2026-08-08 11:46:10.186 UTC [31019] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 11:46:10.186 UTC [31019] LOG: listening on IPv4 address "127.13.158.84", port 39881
2026-08-08 11:46:10.189 UTC [31019] LOG: listening on Unix socket "/tmp/.s.PGSQL.39881"
2026-08-08 11:46:10.212 UTC [31025] LOG: database system was shut down at 2026-08-08 11:46:08 UTC
2026-08-08 11:46:10.216 UTC [31019] LOG: database system is ready to accept connections
I20260808 11:46:11.365582 13945 mini_postgres.cc:96] Postgres bound to 39881
2026-08-08 11:46:11.387 UTC [31040] FATAL: database "slave" does not exist
127.13.158.84:39881 - accepting connections
I20260808 11:46:11.388307 13945 mini_ranger.cc:161] Starting Ranger...
I20260808 11:46:11.418682 13945 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 11:46:11.836746 13945 mini_ranger.cc:87] Created ranger Postgres database
I20260808 11:46:11.836846 13945 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-admin
2026-08-08 11:46:12,106 [I] DB FLAVOR :POSTGRES
2026-08-08 11:46:12,107 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 11:46:12,107 [I] Checking connection..
2026-08-08 11:46:12,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 11:46:12,356 [I] Checking connection passed.
2026-08-08 11:46:12,357 [I] --------- Verifying version history table ---------
2026-08-08 11:46:12,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:46:12,612 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 11:46:12,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:46:12,878 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 11:46:12,879 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 11:46:12,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 11:46:13.141 UTC [31131] WARNING: there is no transaction in progress
2026-08-08 11:46:13,149 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 11:46:13,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:46:13,388 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 11:46:13,389 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 11:46:13,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 11:46:13,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 11:46:13,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-39l8.c.gcp-upstream.internal','N') ;"
2026-08-08 11:46:14,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 11:46:14,418 [I] Table x_portal_user does not exist in database ranger
2026-08-08 11:46:14,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 11:46:14,662 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 11:46:14,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 11:46:14,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 11:46:15,168 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 11:46:15,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 11:46:16.077 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.092 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.107 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.121 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.136 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.152 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.203 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.209 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.222 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.231 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.240 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.252 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.262 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.268 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.273 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16.279 UTC [31286] WARNING: there is no transaction in progress
2026-08-08 11:46:16,834 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 11:46:16,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 11:46:17,078 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 11:46:17,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 11:46:17,345 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 11:46:17,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 11:46:17,590 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 11:46:17,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 11:46:17,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 11:46:18,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-39l8.c.gcp-upstream.internal';"
2026-08-08 11:46:18,337 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 11:46:18,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 11:46:18,596 [I] DB_PATCHES have already been applied
I20260808 11:46:18.602380 13945 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-admin:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskWrISxk/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 11:46:18.602475 13945 mini_ranger.cc:193] Using host: 127.13.158.84
I20260808 11:46:18.621873 13945 mini_ranger.cc:227] Ranger admin URL: http://127.13.158.84:36171
Aug 08, 2026 11:46:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:46:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:46:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:46:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:46:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:46:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:46:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 11:46:19.957875 13945 mini_ranger.cc:160] Time spent starting Ranger: real 8.570s user 0.002s sys 0.051s
Aug 08, 2026 11:46:40 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:46:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 11:46:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:46:40 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:46:41 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 11:46:41.819682 13945 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/rangerkms_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/rangerkms_127.13.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.158.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.158.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.158.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/HTTP_127.13.158.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.158.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/HTTP_127.13.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 11:46:41 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189601, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 11:46:42.004640 13945 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 11:46:42.032322 13945 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 11:46:42.831679 13945 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 11:46:42.831805 13945 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms
I20260808 11:46:42.831857 13945 mini_ranger_kms.cc:227] Using postgres at 127.13.158.84:39881
2026-08-08 11:46:43,237 [I] DB FLAVOR :POSTGRES
2026-08-08 11:46:43,237 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 11:46:43,237 [I] Checking connection
2026-08-08 11:46:43,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 11:46:43,485 [I] connection success
2026-08-08 11:46:43,486 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 11:46:43,486 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 11:46:43,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 11:46:43,748 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 11:46:43,748 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 11:46:43,748 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 11:46:43,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.158.84:39881/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 11:46:44,020 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 11:46:44.499913 13945 mini_ranger_kms.cc:310] Created kms service
I20260808 11:46:44.703153 13945 mini_ranger_kms.cc:326] Created kudu user
I20260808 11:46:44.748920 13945 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 11:46:44 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189601, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:45.146827 13945 mini_ranger_kms.cc:384] Added ranger policy
I20260808 11:46:45.147182 13945 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms:/tmp/dist-test-taskWrISxk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskWrISxk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 11:46:45.147251 13945 mini_ranger_kms.cc:240] Using host: 127.13.158.84
I20260808 11:46:45.170800 13945 mini_ranger_kms.cc:276] Ranger KMS PID: 31568
I20260808 11:46:45.170924 13945 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.158.84:53497
Aug 08, 2026 11:46:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:46:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:46:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:46:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:46:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:46:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 11:46:47 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189607, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.158.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:47.877432 13945 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.873s user 0.069s sys 0.371s
I20260808 11:46:47.877710 13945 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 11:46:47.877899 13945 mini_ranger_kms.cc:401] 127.13.158.84:53497/kms/v1/keys
Aug 08 11:46:47 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189607, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.158.84@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
Aug 08 11:46:48 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189607, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.158.84@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
Aug 08 11:46:50 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:46:50.594950 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:39607
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:39607
--ranger_config_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:53497/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:46:50.942162 31634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:50.942431 31634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:50.942569 31634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:50.949488 31634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:46:50.949580 31634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:50.949656 31634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:50.949728 31634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:46:50.949808 31634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:46:50.959687 31634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.158.84:53497/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:39607
--ranger_config_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:50.960862 31634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:50.962215 31634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:50.975350 31644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:50.975785 31642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:50.975957 31646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:50.977306 31634 server_base.cc:1034] running on GCE node
I20260808 11:46:50.978134 31634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:50.979493 31634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:50.980721 31634 hybrid_clock.cc:648] HybridClock initialized: now 1786189610980673 us; error 44 us; skew 500 ppm
I20260808 11:46:50.981087 31634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:50 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189610, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:50.984297 31634 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:46:50.985852 31634 webserver.cc:460] Webserver started at http://127.13.158.126:42577/ using document root <none> and password file <none>
I20260808 11:46:50.986522 31634 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:50.986641 31634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:46:50.986970 31634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:46:50 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189610, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:51.046154 31634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "9ca8a61029a34e74a09687a7cecdd934"
format_stamp: "Formatted at 2026-08-08 11:46:51 on dist-test-slave-39l8"
server_key: "799eca1c9c0d240072b97428b5c6db72"
server_key_iv: "c94a8be34669487c503450e888d5f4ec"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:51.047091 31634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "9ca8a61029a34e74a09687a7cecdd934"
format_stamp: "Formatted at 2026-08-08 11:46:51 on dist-test-slave-39l8"
server_key: "799eca1c9c0d240072b97428b5c6db72"
server_key_iv: "c94a8be34669487c503450e888d5f4ec"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:51.052534 31634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:46:51.071381 31656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:51.072914 31634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:46:51.073114 31634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "9ca8a61029a34e74a09687a7cecdd934"
format_stamp: "Formatted at 2026-08-08 11:46:51 on dist-test-slave-39l8"
server_key: "799eca1c9c0d240072b97428b5c6db72"
server_key_iv: "c94a8be34669487c503450e888d5f4ec"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:51.073345 31634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:51.085152 31634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:51.088800 31634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:51.089104 31634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:51.109493 31634 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:39607
I20260808 11:46:51.109501 31732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:39607 every 8 connection(s)
I20260808 11:46:51.110947 31634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:46:51.114286 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 31634
I20260808 11:46:51.114588 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:46:51.115147 31733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:46:51.115581 13945 external_mini_cluster.cc:1428] Setting key 53b4e036b6270e2a58935e029fecf158
I20260808 11:46:51.132956 31733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Bootstrap starting.
I20260808 11:46:51.136059 31733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Neither blocks nor log segments found. Creating new log.
I20260808 11:46:51.137295 31733 log.cc:824] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:51.140386 31733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: No bootstrap required, opened a new log
Aug 08 11:46:51 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:51.148430 31733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } }
I20260808 11:46:51.148837 31733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:46:51.148998 31733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ca8a61029a34e74a09687a7cecdd934, State: Initialized, Role: FOLLOWER
I20260808 11:46:51.149655 31733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } }
I20260808 11:46:51.149961 31733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:46:51.150137 31733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:46:51.150352 31733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:46:51.152808 31733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } }
I20260808 11:46:51.153409 31733 leader_election.cc:302] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ca8a61029a34e74a09687a7cecdd934; no voters:
I20260808 11:46:51.154091 31733 leader_election.cc:288] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:46:51.154176 31741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:46:51.156747 31741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 1 LEADER]: Becoming Leader. State: Replica: 9ca8a61029a34e74a09687a7cecdd934, State: Running, Role: LEADER
I20260808 11:46:51.156811 31733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:46:51.157294 31741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } }
I20260808 11:46:51.159858 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:51.117669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.13.158.126:39607)
0808 11:46:51.135276 (+ 17607us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:51.135288 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:51.135330 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:46:51.137122 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:51.137145 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:51.137554 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:51.137994 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:51.139597 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:51.141447 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:51.143184 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:51.144133 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:51.144309 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:51.149049 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:51.149058 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:51.149074 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:51.149115 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:51.152862 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:51.153798 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:51.153805 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:51.153812 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:51.153975 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:51.154446 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:51.154450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:51.154471 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:51.156208 (+ 1737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:51.156548 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:51.157714 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0808 11:46:51.158147 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17387,"thread_start_us":17153,"threads_started":1}
I20260808 11:46:51.168176 31742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ca8a61029a34e74a09687a7cecdd934" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } } }
I20260808 11:46:51.168514 31742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: This master's current role is: LEADER
I20260808 11:46:51.181648 31743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ca8a61029a34e74a09687a7cecdd934. Latest consensus state: current_term: 1 leader_uuid: "9ca8a61029a34e74a09687a7cecdd934" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } } }
I20260808 11:46:51.181955 31743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: This master's current role is: LEADER
I20260808 11:46:51.182253 31753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:46:51.188678 31753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:46:51.189374 31733 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 11:46:51.211124 31753 catalog_manager.cc:1347] Generated new cluster ID: a573db307c344592b19ecd7f8f213d00
I20260808 11:46:51.211294 31753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:46:51.243191 31753 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:46:51.245956 31753 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:46:51.255110 31753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Generated new TSK 0
I20260808 11:46:51.256287 31753 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:46:51.987985 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:53497/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39607
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:46:52 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 11:46:52.414342 31808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:52.414822 31808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:52.415040 31808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:52.426455 31808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:52.426635 31808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:52.426862 31808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:46:52.442263 31808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.158.84:53497/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:52.444104 31808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:52.445904 31808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:52.463708 31827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:52.465804 31828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:52.471141 31808 server_base.cc:1034] running on GCE node
W20260808 11:46:52.471884 31830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:52.472592 31808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:52.473698 31808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:52.474977 31808 hybrid_clock.cc:648] HybridClock initialized: now 1786189612474893 us; error 79 us; skew 500 ppm
I20260808 11:46:52.475315 31808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:52 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:52.479166 31808 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:46:52.498116 31808 webserver.cc:460] Webserver started at http://127.13.158.65:35257/ using document root <none> and password file <none>
I20260808 11:46:52.499289 31808 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:52.499452 31808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:46:52.499786 31808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:46:52 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:52.515937 31808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "56cc32be28ba4af6ab877f0db54b2187"
format_stamp: "Formatted at 2026-08-08 11:46:52 on dist-test-slave-39l8"
server_key: "d44de4e1cfa01f4ea78f7a4b660a2da3"
server_key_iv: "2ac3c602c918517128066af95b4aa603"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:52.516772 31808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "56cc32be28ba4af6ab877f0db54b2187"
format_stamp: "Formatted at 2026-08-08 11:46:52 on dist-test-slave-39l8"
server_key: "d44de4e1cfa01f4ea78f7a4b660a2da3"
server_key_iv: "2ac3c602c918517128066af95b4aa603"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:52.523691 31808 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260808 11:46:52.543396 31841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:52.544978 31808 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.004s
I20260808 11:46:52.545188 31808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "56cc32be28ba4af6ab877f0db54b2187"
format_stamp: "Formatted at 2026-08-08 11:46:52 on dist-test-slave-39l8"
server_key: "d44de4e1cfa01f4ea78f7a4b660a2da3"
server_key_iv: "2ac3c602c918517128066af95b4aa603"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:52.545414 31808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:52.576001 31808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:52.581441 31808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:52.581882 31808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:52.587106 31808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:52.588912 31808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:52.589015 31808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:52.589149 31808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:52.589232 31808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:52.672283 31808 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:40579
I20260808 11:46:52.672343 32005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:40579 every 8 connection(s)
I20260808 11:46:52.673744 31808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:46:52.678687 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 31808
I20260808 11:46:52.678920 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:46:52.679783 13945 external_mini_cluster.cc:1428] Setting key fe67cecbe58a35648da550614c200789
Aug 08 11:46:52 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
I20260808 11:46:52.719553 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:52.677002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:45567 (local address 127.13.158.126:39607)
0808 11:46:52.692780 (+ 15778us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:52.692788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:52.692812 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:46:52.692880 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:52.692888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:52.692992 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:52.693123 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:52.695274 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:52.696428 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:52.698788 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:52.699380 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:52.699523 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:52.704184 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:52.704194 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:52.704199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:52.704243 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:52.710695 (+ 6452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:52.712203 (+ 1508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:52.712209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:52.712213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:52.712375 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:52.712958 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:52.712963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:52.712967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:52.713358 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:52.714622 (+ 1264us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:52.717602 (+ 2980us) server_negotiation.cc:299] Negotiation successful
0808 11:46:52.718949 (+ 1347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15621,"thread_start_us":15502,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
I20260808 11:46:52.722002 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:52.677803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39607 (local address 127.13.158.65:45567)
0808 11:46:52.678657 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:52.678737 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:52.682666 (+ 3929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:52.693450 (+ 10784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:52.693463 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:52.694181 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:52.695074 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:52.695103 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:52.696584 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:52.696590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:52.698583 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:52.698612 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:52.700051 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:52.700059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:52.700353 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:52.701122 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:52.701150 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:52.703898 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:52.711347 (+ 7449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:52.711358 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:52.711377 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:52.712030 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:52.712536 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:52.712541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:52.712545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:52.712812 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:52.714800 (+ 1988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:52.714809 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:52.715288 (+ 479us) client_negotiation.cc:769] Sending connection context
0808 11:46:52.719955 (+ 4667us) client_negotiation.cc:240] Negotiation successful
0808 11:46:52.720557 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":3061,"threads_started":1}
I20260808 11:46:52.724328 32006 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39607
I20260808 11:46:52.724696 32006 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:52.725562 32006 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
I20260808 11:46:52.728188 31685 ts_manager.cc:194] Registered new tserver with Master: 56cc32be28ba4af6ab877f0db54b2187 (127.13.158.65:40579)
I20260808 11:46:52.730161 31685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:45567
Aug 08 11:46:52 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:46:52.766901 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:53497/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39607
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:46:53 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
W20260808 11:46:53.243062 32016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:53.243618 32016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:53.243844 32016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:53.256131 32016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:53.256351 32016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:53.256594 32016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:46:53.272102 32016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.158.84:53497/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:53.274112 32016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:53.276031 32016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:53.308846 32026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:53.312554 32025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:53.315068 32016 server_base.cc:1034] running on GCE node
W20260808 11:46:53.315790 32028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:53.316668 32016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:53.317965 32016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:53.319258 32016 hybrid_clock.cc:648] HybridClock initialized: now 1786189613319198 us; error 49 us; skew 500 ppm
I20260808 11:46:53.319689 32016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:53 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:53.324046 32016 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:46:53.326053 32016 webserver.cc:460] Webserver started at http://127.13.158.66:45323/ using document root <none> and password file <none>
I20260808 11:46:53.327112 32016 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:53.327286 32016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:46:53.327677 32016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:46:53 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:53.344422 32016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "99e9ccdef60548068121bb3fa725dd40"
format_stamp: "Formatted at 2026-08-08 11:46:53 on dist-test-slave-39l8"
server_key: "3b28d78b3e0d1fb1c8503e5602fea7ea"
server_key_iv: "8ad280fc8fa276ddd0c26ea10ce9eab7"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:53.345522 32016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "99e9ccdef60548068121bb3fa725dd40"
format_stamp: "Formatted at 2026-08-08 11:46:53 on dist-test-slave-39l8"
server_key: "3b28d78b3e0d1fb1c8503e5602fea7ea"
server_key_iv: "8ad280fc8fa276ddd0c26ea10ce9eab7"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:53.361464 32016 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.004s sys 0.005s
I20260808 11:46:53.391323 32040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:53.394531 32016 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.001s sys 0.007s
I20260808 11:46:53.394905 32016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "99e9ccdef60548068121bb3fa725dd40"
format_stamp: "Formatted at 2026-08-08 11:46:53 on dist-test-slave-39l8"
server_key: "3b28d78b3e0d1fb1c8503e5602fea7ea"
server_key_iv: "8ad280fc8fa276ddd0c26ea10ce9eab7"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:53.395298 32016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:53.424839 32016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:53.429147 32016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:53.429478 32016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:53.430583 32016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:53.432106 32016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:53.432191 32016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:53.432369 32016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:53.432420 32016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:53.470352 32016 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:36633
I20260808 11:46:53.470371 32201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:36633 every 8 connection(s)
I20260808 11:46:53.471784 32016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:46:53.479259 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 32016
I20260808 11:46:53.479449 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:46:53.480085 13945 external_mini_cluster.cc:1428] Setting key 1102fda11427359be27a147c28d48dc0
Aug 08 11:46:53 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:53.495229 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:53.475029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:52709 (local address 127.13.158.126:39607)
0808 11:46:53.475570 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:53.475577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:53.476739 (+ 1162us) server_negotiation.cc:407] Connection header received
0808 11:46:53.478738 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:53.478744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:53.478849 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:53.478978 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:53.481065 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:53.482183 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:53.484571 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:53.485091 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:53.485178 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:53.488807 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:53.488812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:53.488815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:53.488864 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:53.491788 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:53.492531 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:53.492538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:53.492546 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:53.492671 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:53.493138 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:53.493147 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:53.493149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:53.493423 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:53.493595 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:53.494407 (+ 812us) server_negotiation.cc:299] Negotiation successful
0808 11:46:53.494837 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":276,"threads_started":1}
I20260808 11:46:53.496294 32208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:53.475666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39607 (local address 127.13.158.66:52709)
0808 11:46:53.476570 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:53.476614 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:53.478440 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:53.479288 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:53.479300 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:53.479914 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:53.480755 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:53.480783 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:53.482388 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:53.482393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:53.484331 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:53.484387 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:53.485445 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:53.485450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:53.485661 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:53.486338 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:53.486360 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:53.488573 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:53.491938 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:53.491946 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:53.491962 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:53.492397 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:53.492784 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:53.492789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:53.492793 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:53.493002 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:53.493577 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:53.493584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:53.494008 (+ 424us) client_negotiation.cc:769] Sending connection context
0808 11:46:53.494295 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 11:46:53.494847 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":378,"threads_started":1}
I20260808 11:46:53.497897 32202 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39607
I20260808 11:46:53.498229 32202 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:53.499022 32202 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
I20260808 11:46:53.500576 31685 ts_manager.cc:194] Registered new tserver with Master: 99e9ccdef60548068121bb3fa725dd40 (127.13.158.66:36633)
I20260808 11:46:53.501606 31685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:52709
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:46:53.543947 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:53497/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39607
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:46:53.733552 32006 heartbeater.cc:503] Master 127.13.158.126:39607 was elected leader, sending a full tablet report...
W20260808 11:46:53.881253 32211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:53.881515 32211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:53.881686 32211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:53.890533 32211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:53.890681 32211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:53.890901 32211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:46:53.903601 32211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.158.84:53497/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:53.904909 32211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:53.906183 32211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:53.919317 32217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:53.919269 32224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:53.919358 32219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:53.920110 32211 server_base.cc:1034] running on GCE node
I20260808 11:46:53.920717 32211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:53.921667 32211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:53.922886 32211 hybrid_clock.cc:648] HybridClock initialized: now 1786189613922848 us; error 52 us; skew 500 ppm
I20260808 11:46:53.923210 32211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:53 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:53.926302 32211 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:46:53.927845 32211 webserver.cc:460] Webserver started at http://127.13.158.67:35515/ using document root <none> and password file <none>
I20260808 11:46:53.928565 32211 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:53.928731 32211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:46:53.929049 32211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:46:53 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:53.944108 32211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "392ceda019eb4e7ab957dfa2528ee04f"
format_stamp: "Formatted at 2026-08-08 11:46:53 on dist-test-slave-39l8"
server_key: "1b16b7b89e441517f19a2440b0d38f3f"
server_key_iv: "6631595cd6ba47c99f57ee5092571405"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:53.944998 32211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "392ceda019eb4e7ab957dfa2528ee04f"
format_stamp: "Formatted at 2026-08-08 11:46:53 on dist-test-slave-39l8"
server_key: "1b16b7b89e441517f19a2440b0d38f3f"
server_key_iv: "6631595cd6ba47c99f57ee5092571405"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:53.950898 32211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:46:53.965760 32238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:53.967339 32211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:46:53.967505 32211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "392ceda019eb4e7ab957dfa2528ee04f"
format_stamp: "Formatted at 2026-08-08 11:46:53 on dist-test-slave-39l8"
server_key: "1b16b7b89e441517f19a2440b0d38f3f"
server_key_iv: "6631595cd6ba47c99f57ee5092571405"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:53.967762 32211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:54.002688 32211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:54.006635 32211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:54.006995 32211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:54.007997 32211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:54.009402 32211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:54.009516 32211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:54.009665 32211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:54.009744 32211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:54.047931 32211 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:39305
I20260808 11:46:54.047969 32397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:39305 every 8 connection(s)
I20260808 11:46:54.049372 32211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:46:54.050484 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 32211
I20260808 11:46:54.050668 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:46:54.051319 13945 external_mini_cluster.cc:1428] Setting key 313c9d92b46e3f3ddbb00e6a9af9a515
Aug 08 11:46:54 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:54.070006 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:54.052503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:58337 (local address 127.13.158.126:39607)
0808 11:46:54.053269 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:54.053276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:54.054067 (+ 791us) server_negotiation.cc:407] Connection header received
0808 11:46:54.056080 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:54.056086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:54.056168 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:54.056298 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:54.057787 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.058636 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.060148 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.060573 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.060721 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:54.064017 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:54.064022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:54.064026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:54.064057 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:54.067464 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:54.068115 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:54.068119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:54.068121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:54.068247 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:54.068635 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:54.068638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:54.068639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:54.068911 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:54.069020 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:54.069593 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 11:46:54.069760 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":817,"threads_started":1}
I20260808 11:46:54.071127 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:54.053116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39607 (local address 127.13.158.67:58337)
0808 11:46:54.053931 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:54.053967 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:54.055833 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:54.056531 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:54.056539 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:54.057031 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:54.057626 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:54.057642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.058817 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.058829 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:54.059996 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:54.060008 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.060797 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.060803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:54.061050 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:54.061660 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:54.061680 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:54.063822 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:54.067588 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:54.067594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:54.067605 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:54.068015 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:54.068338 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:54.068341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:54.068344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:54.068536 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:54.069016 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:54.069021 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:54.069304 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 11:46:54.069515 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 11:46:54.069884 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":365,"threads_started":1}
I20260808 11:46:54.072538 32402 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39607
I20260808 11:46:54.072880 32402 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:54.073602 32402 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
I20260808 11:46:54.075093 31685 ts_manager.cc:194] Registered new tserver with Master: 392ceda019eb4e7ab957dfa2528ee04f (127.13.158.67:39305)
I20260808 11:46:54.075923 31685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:58337
I20260808 11:46:54.079511 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:46:54.081296 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 31808
Aug 08 11:46:54 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:46:54.098022 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:54.082268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.13.158.65:40579)
0808 11:46:54.082977 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:54.082983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:54.083007 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:46:54.083061 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:54.083067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:54.083240 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:54.083365 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:54.084847 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.085897 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.087504 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.088047 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.088143 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:54.092306 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:54.092319 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:54.092332 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:54.092370 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:54.095542 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:54.096248 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:54.096255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:54.096257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:54.096401 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:54.096722 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:54.096725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:54.096727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:54.097030 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:54.097255 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:54.097517 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 11:46:54.097743 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":379,"threads_started":1}
I20260808 11:46:54.099366 31999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:54.183424 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 31808
I20260808 11:46:54.207973 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 32016
Aug 08 11:46:54 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:46:54.224023 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:54.208938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52058 (local address 127.13.158.66:36633)
0808 11:46:54.209503 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:54.209510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:54.209538 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:46:54.209603 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:54.209609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:54.209828 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:54.209963 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:54.211331 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.212327 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.213629 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.214167 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.214264 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:54.218584 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:54.218596 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:54.218611 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:54.218652 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:54.221346 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:54.221994 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:54.222001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:54.222003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:54.222129 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:54.222441 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:54.222444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:54.222446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:54.222951 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:54.223203 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:54.223427 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 11:46:54.223616 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":296,"threads_started":1}
I20260808 11:46:54.225288 32194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:54.308061 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 32016
I20260808 11:46:54.325742 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 32211
Aug 08 11:46:54 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:46:54.341920 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:54.326772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.13.158.67:39305)
0808 11:46:54.327419 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:54.327426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:54.327454 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:46:54.327512 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:54.327518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:54.327727 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:54.327899 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:54.329309 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.330341 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.331782 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:54.332349 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:54.332487 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:54.336922 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:54.336932 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:54.336946 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:54.336977 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:54.339610 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:54.340291 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:54.340298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:54.340300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:54.340422 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:54.340743 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:54.340746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:54.340748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:54.341059 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:54.341298 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:54.341470 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 11:46:54.341647 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":342,"threads_started":1}
I20260808 11:46:54.343250 32393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:54.426141 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 32211
I20260808 11:46:54.447141 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 31634
I20260808 11:46:54.448433 31726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:54.552006 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 31634
I20260808 11:46:54.585619 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:39607
--webserver_interface=127.13.158.126
--webserver_port=42577
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:39607
--ranger_config_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:53497/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.158.84:53497/kms with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:46:54.921905 32418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:54.922184 32418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:54.922272 32418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:54.929256 32418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:46:54.929363 32418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:54.929409 32418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:54.929463 32418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:46:54.929512 32418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:46:54.939460 32418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.158.84:53497/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:39607
--ranger_config_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=42577
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:54.940632 32418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:54.942005 32418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:54.954492 32428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:54.954499 32425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:54.954813 32424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:54.955282 32418 server_base.cc:1034] running on GCE node
I20260808 11:46:54.956065 32418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:54.957393 32418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:54.958598 32418 hybrid_clock.cc:648] HybridClock initialized: now 1786189614958546 us; error 49 us; skew 500 ppm
I20260808 11:46:54.958985 32418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:54 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:54.962070 32418 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:46:54.963644 32418 webserver.cc:460] Webserver started at http://127.13.158.126:42577/ using document root <none> and password file <none>
I20260808 11:46:54.964254 32418 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:54.964365 32418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 11:46:54 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:55.014109 32418 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:46:55.017351 32440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:55.018949 32418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 11:46:55.019145 32418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "9ca8a61029a34e74a09687a7cecdd934"
format_stamp: "Formatted at 2026-08-08 11:46:51 on dist-test-slave-39l8"
server_key: "799eca1c9c0d240072b97428b5c6db72"
server_key_iv: "c94a8be34669487c503450e888d5f4ec"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:55.020042 32418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:55.035910 32418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:55.039469 32418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:55.039776 32418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:55.058240 32418 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:39607
I20260808 11:46:55.058248 32512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:39607 every 8 connection(s)
I20260808 11:46:55.059787 32418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:46:55.064153 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 32418
I20260808 11:46:55.065248 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:53497/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:40579
--local_ip_for_outbound_sockets=127.13.158.65
--tserver_master_addrs=127.13.158.126:39607
--webserver_port=35257
--webserver_interface=127.13.158.65
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.158.84:53497/kms with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:46:55.067843 32513 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:46:55.072804 32513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Bootstrap starting.
I20260808 11:46:55.092479 32513 log.cc:824] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:55.101393 32513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:46:55.101949 32513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Bootstrap complete.
I20260808 11:46:55.111073 32513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } }
I20260808 11:46:55.111939 32513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ca8a61029a34e74a09687a7cecdd934, State: Initialized, Role: FOLLOWER
I20260808 11:46:55.112715 32513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } }
I20260808 11:46:55.112957 32513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:46:55.113092 32513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:46:55.113297 32513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:46:55.117228 32513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } }
I20260808 11:46:55.117710 32513 leader_election.cc:302] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ca8a61029a34e74a09687a7cecdd934; no voters:
I20260808 11:46:55.118340 32513 leader_election.cc:288] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 11:46:55.118477 32517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:46:55.119874 32517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [term 2 LEADER]: Becoming Leader. State: Replica: 9ca8a61029a34e74a09687a7cecdd934, State: Running, Role: LEADER
I20260808 11:46:55.120615 32513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:46:55.120487 32517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } }
I20260808 11:46:55.123248 32518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9ca8a61029a34e74a09687a7cecdd934" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } } }
I20260808 11:46:55.123684 32518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: This master's current role is: LEADER
I20260808 11:46:55.128509 32519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ca8a61029a34e74a09687a7cecdd934. Latest consensus state: current_term: 2 leader_uuid: "9ca8a61029a34e74a09687a7cecdd934" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca8a61029a34e74a09687a7cecdd934" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 39607 } } }
I20260808 11:46:55.128855 32519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934 [sys.catalog]: This master's current role is: LEADER
I20260808 11:46:55.131517 32527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:46:55.137287 32527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:46:55.138659 32527 catalog_manager.cc:1259] Loaded cluster ID: a573db307c344592b19ecd7f8f213d00
I20260808 11:46:55.138839 32527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:46:55.142868 32527 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:46:55.144497 32527 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Loaded TSK: 0
I20260808 11:46:55.146165 32527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:46:55.151424 32513 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 11:46:55.490921 32515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:55.491297 32515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:55.491466 32515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:55.502323 32515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:55.502537 32515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:55.502799 32515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:46:55.517514 32515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.158.84:53497/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:40579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=35257
--tserver_master_addrs=127.13.158.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:55.519140 32515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:55.520712 32515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:55.556279 32563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:55.557066 32565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:55.565050 32568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:55.565343 32515 server_base.cc:1034] running on GCE node
I20260808 11:46:55.566377 32515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:55.567873 32515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:55.571352 32515 hybrid_clock.cc:648] HybridClock initialized: now 1786189615571253 us; error 63 us; skew 500 ppm
I20260808 11:46:55.571887 32515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:55 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189615, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:55.577522 32515 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:46:55.584860 32515 webserver.cc:460] Webserver started at http://127.13.158.65:35257/ using document root <none> and password file <none>
I20260808 11:46:55.585664 32515 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:55.585843 32515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 11:46:55 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189615, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:55.625430 32515 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.000s
I20260808 11:46:55.635515 32577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:55.637595 32515 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.007s
I20260808 11:46:55.637856 32515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "56cc32be28ba4af6ab877f0db54b2187"
format_stamp: "Formatted at 2026-08-08 11:46:52 on dist-test-slave-39l8"
server_key: "d44de4e1cfa01f4ea78f7a4b660a2da3"
server_key_iv: "2ac3c602c918517128066af95b4aa603"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:55.639575 32515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:55.681913 32515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:55.687269 32515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:55.687674 32515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:55.699173 32515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:55.715664 32515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:55.715864 32515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:55.716060 32515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:55.716197 32515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:55.809731 32515 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:40579
I20260808 11:46:55.809780 32746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:40579 every 8 connection(s)
I20260808 11:46:55.811358 32515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:46:55.819695 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 32515
I20260808 11:46:55.820745 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:53497/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:36633
--local_ip_for_outbound_sockets=127.13.158.66
--tserver_master_addrs=127.13.158.126:39607
--webserver_port=45323
--webserver_interface=127.13.158.66
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.158.84:53497/kms with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:46:55 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189615, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:55.851806 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:55.815276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39607 (local address 127.13.158.65:50965)
0808 11:46:55.826506 (+ 11230us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:55.826562 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:55.830187 (+ 3625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:55.833203 (+ 3016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:55.833216 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:55.833883 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:55.834671 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:55.834700 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:55.836765 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:55.836771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:55.838693 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:55.838710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:55.840407 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:55.840415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:55.840646 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:55.841354 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:55.841381 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:55.843388 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:55.847191 (+ 3803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:55.847202 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:55.847220 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:55.847629 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:55.848169 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:55.848173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:55.848177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:55.848365 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:55.849082 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:55.849091 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:55.849408 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 11:46:55.849670 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 11:46:55.850194 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10874,"thread_start_us":12763,"threads_started":1}
I20260808 11:46:55.851781 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:55.815229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:50965 (local address 127.13.158.126:39607)
0808 11:46:55.831081 (+ 15852us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:55.831103 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:55.831156 (+ 53us) server_negotiation.cc:407] Connection header received
0808 11:46:55.831589 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:55.831610 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:55.832085 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:55.832481 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:55.834935 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:55.836588 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:55.838917 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:55.839828 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:55.840020 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:55.843651 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:55.843658 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:55.843669 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:55.843708 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:55.847010 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:55.847794 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:55.847803 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:55.847810 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:55.848015 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:55.848494 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:55.848498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:55.848500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:55.848938 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:55.849152 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:55.849877 (+ 725us) server_negotiation.cc:299] Negotiation successful
0808 11:46:55.850199 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15525,"thread_start_us":15278,"threads_started":1}
I20260808 11:46:55.855271 32748 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39607
I20260808 11:46:55.855597 32748 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:55.856411 32748 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
W20260808 11:46:55.859609 32748 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:39607 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 11:46:55.859833 32748 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:55.860100 32748 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
I20260808 11:46:55.861296 32748 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:55.861627 32748 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
W20260808 11:46:55.862812 32748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:46:56.234901 32753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:56.235282 32753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:56.235452 32753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:56.244480 32753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:56.244632 32753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:56.244830 32753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:46:56.260447 32753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.158.84:53497/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:36633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=45323
--tserver_master_addrs=127.13.158.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:56.262257 32753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:56.264001 32753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:56.287876 301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:56.287909 303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:56.288925 306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:56.289661 32753 server_base.cc:1034] running on GCE node
I20260808 11:46:56.290380 32753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:56.291657 32753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:56.292920 32753 hybrid_clock.cc:648] HybridClock initialized: now 1786189616292860 us; error 44 us; skew 500 ppm
I20260808 11:46:56.293320 32753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:56 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:56.297488 32753 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:46:56.326709 32753 webserver.cc:460] Webserver started at http://127.13.158.66:45323/ using document root <none> and password file <none>
I20260808 11:46:56.328008 32753 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:56.328271 32753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 11:46:56 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:56.372845 32753 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.003s
I20260808 11:46:56.376533 327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:56.378316 32753 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260808 11:46:56.378494 32753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "99e9ccdef60548068121bb3fa725dd40"
format_stamp: "Formatted at 2026-08-08 11:46:53 on dist-test-slave-39l8"
server_key: "3b28d78b3e0d1fb1c8503e5602fea7ea"
server_key_iv: "8ad280fc8fa276ddd0c26ea10ce9eab7"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:56.379863 32753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:56.399843 32753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:56.404949 32753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:56.405256 32753 kserver.cc:163] Server-wide thread pool size limit: 3276
Aug 08 11:46:56 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:56.406534 32753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:56.408285 32753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:56.408429 32753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:56.408555 32753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:56.408658 32753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:56.495550 32753 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:36633
I20260808 11:46:56.495620 494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:36633 every 8 connection(s)
I20260808 11:46:56.497573 32753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:46:56.502045 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 32753
I20260808 11:46:56.502972 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.158.84:53497/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:39305
--local_ip_for_outbound_sockets=127.13.158.67
--tserver_master_addrs=127.13.158.126:39607
--webserver_port=35515
--webserver_interface=127.13.158.67
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.158.84:53497/kms with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:46:56 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:56.537364 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:56.500905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:57339 (local address 127.13.158.126:39607)
0808 11:46:56.501436 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:56.501443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:56.514080 (+ 12637us) server_negotiation.cc:407] Connection header received
0808 11:46:56.514799 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:56.514810 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:56.514919 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:56.515035 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:56.517183 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:56.518551 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:56.521386 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:56.522285 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:56.522401 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:56.527746 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:56.527756 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:56.527765 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:56.527826 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:56.532318 (+ 4492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:56.533591 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:56.533601 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:56.533608 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:56.533789 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:56.534547 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:56.534555 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:56.534560 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:56.535079 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:56.535252 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:56.536490 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0808 11:46:56.536784 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":1361,"threads_started":1}
I20260808 11:46:56.540613 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:56.501945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39607 (local address 127.13.158.66:57339)
0808 11:46:56.511353 (+ 9408us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:56.511428 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:56.514388 (+ 2960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:56.515700 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:56.515715 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:56.516274 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:56.516988 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:56.517019 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:56.518777 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:56.518786 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:56.521190 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:56.521220 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:56.522828 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:56.522838 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:56.523176 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:56.524065 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:56.524104 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:56.527421 (+ 3317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:56.532520 (+ 5099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:56.532531 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:56.532554 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:56.533426 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:56.533975 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:56.533982 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:56.533990 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:56.534379 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:56.535371 (+ 992us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:56.535383 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:56.535928 (+ 545us) client_negotiation.cc:769] Sending connection context
0808 11:46:56.538006 (+ 2078us) client_negotiation.cc:240] Negotiation successful
0808 11:46:56.538771 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9051,"thread_start_us":8799,"threads_started":1}
I20260808 11:46:56.542969 495 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39607
I20260808 11:46:56.543322 495 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:56.544200 495 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
I20260808 11:46:56.546370 32465 ts_manager.cc:194] Registered new tserver with Master: 99e9ccdef60548068121bb3fa725dd40 (127.13.158.66:36633)
I20260808 11:46:56.548751 32465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:57339
I20260808 11:46:56.862952 32748 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:56.863415 32748 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
I20260808 11:46:56.864835 32465 ts_manager.cc:194] Registered new tserver with Master: 56cc32be28ba4af6ab877f0db54b2187 (127.13.158.65:40579)
I20260808 11:46:56.866036 32465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:50965
Aug 08 11:46:56 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
W20260808 11:46:56.949529 502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:46:56.949906 502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:46:56.950069 502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:46:56.961421 502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:46:56.961551 502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:46:56.961712 502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:46:56.976701 502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.158.84:53497/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:39305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=35515
--tserver_master_addrs=127.13.158.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:46:56.978386 502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:46:56.980033 502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:46:56.997772 508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:56.999358 510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:46:56.999773 513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:46:57.001291 502 server_base.cc:1034] running on GCE node
I20260808 11:46:57.002126 502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:46:57.003360 502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:46:57.004601 502 hybrid_clock.cc:648] HybridClock initialized: now 1786189617004540 us; error 55 us; skew 500 ppm
I20260808 11:46:57.004964 502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:46:57 dist-test-slave-39l8 krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:46:57.009161 502 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:46:57.011127 502 webserver.cc:460] Webserver started at http://127.13.158.67:35515/ using document root <none> and password file <none>
I20260808 11:46:57.012148 502 fs_manager.cc:359] Metadata directory not provided
I20260808 11:46:57.012324 502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 11:46:57 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
I20260808 11:46:57.038494 502 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20260808 11:46:57.042179 523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:57.044000 502 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 11:46:57.044194 502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "392ceda019eb4e7ab957dfa2528ee04f"
format_stamp: "Formatted at 2026-08-08 11:46:53 on dist-test-slave-39l8"
server_key: "1b16b7b89e441517f19a2440b0d38f3f"
server_key_iv: "6631595cd6ba47c99f57ee5092571405"
server_key_version: "kuduclusterkey@0"
I20260808 11:46:57.045778 502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:46:57.081804 502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:46:57.087070 502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:46:57.087415 502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:46:57.088639 502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:46:57.090533 502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:46:57.090654 502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:57.090782 502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:46:57.090853 502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:46:57.170059 691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:39305 every 8 connection(s)
I20260808 11:46:57.170450 502 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:39305
I20260808 11:46:57.172298 502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:46:57.178154 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 502
Aug 08 11:46:57 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for HTTP/127.13.158.84@KRBTEST.COM
Aug 08 11:46:57 dist-test-slave-39l8 krb5kdc[30977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:46:57.227906 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.179305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.13.158.126:39607)
0808 11:46:57.198204 (+ 18899us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.198210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.198233 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:46:57.198292 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.198298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.198391 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.198496 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:57.200817 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.201923 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.204039 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.204686 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.204813 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.211678 (+ 6865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:57.211686 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:57.211691 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:57.211748 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:57.219591 (+ 7843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:57.223041 (+ 3450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:57.223047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:57.223050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:57.223268 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:57.225566 (+ 2298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:57.225572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:57.225575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:57.225958 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:57.226276 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.227297 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.227539 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18770,"thread_start_us":18671,"threads_started":1}
I20260808 11:46:57.235484 697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.185145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:39607 (local address 127.13.158.67:39233)
0808 11:46:57.193435 (+ 8290us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:57.193513 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:57.195962 (+ 2449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:57.206879 (+ 10917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:57.206892 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:57.207600 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:46:57.208511 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.208539 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.219917 (+ 11378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.219924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.221712 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.221735 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.223358 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.223367 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.223607 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.224380 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:46:57.224417 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:46:57.227080 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:46:57.230757 (+ 3677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:57.230765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:57.230783 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:57.231330 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:57.231777 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:46:57.231782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:46:57.231785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:46:57.232122 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:46:57.232765 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:46:57.232773 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:46:57.233230 (+ 457us) client_negotiation.cc:769] Sending connection context
0808 11:46:57.233512 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 11:46:57.234079 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7960,"thread_start_us":11119,"threads_started":1}
I20260808 11:46:57.236552 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.198251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:39233 (local address 127.13.158.126:39607)
0808 11:46:57.206230 (+ 7979us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.206236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.206255 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:46:57.206299 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.206303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.206378 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.206479 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:57.218515 (+ 12036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.219711 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.221939 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.222669 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.222826 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.227379 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:57.227385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:57.227389 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:57.227425 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:57.230598 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:57.231519 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:57.231524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:57.231527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:57.231657 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:57.232255 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:57.232260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:57.232263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:57.232568 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:57.232686 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.236001 (+ 3315us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.236240 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7849,"thread_start_us":8358,"threads_started":1}
I20260808 11:46:57.241077 692 heartbeater.cc:347] Connected to a master server at 127.13.158.126:39607
I20260808 11:46:57.241415 692 heartbeater.cc:464] Registering TS with master...
I20260808 11:46:57.242328 692 heartbeater.cc:511] Master 127.13.158.126:39607 requested a full tablet report, sending...
I20260808 11:46:57.244058 32465 ts_manager.cc:194] Registered new tserver with Master: 392ceda019eb4e7ab957dfa2528ee04f (127.13.158.67:39305)
I20260808 11:46:57.244946 32465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:39233
I20260808 11:46:57.256701 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:46:57.342701 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.310866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.13.158.126:39607)
0808 11:46:57.311059 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.311066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.315351 (+ 4285us) server_negotiation.cc:407] Connection header received
0808 11:46:57.315668 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.315675 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.315770 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.315889 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:57.319409 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.320572 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.324337 (+ 3765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.325030 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.325136 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.332571 (+ 7435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:57.332579 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:57.332584 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:57.332633 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:57.336341 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:57.340297 (+ 3956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:57.340303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:57.340307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:57.340461 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:57.340949 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:57.340954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:57.340958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:57.341340 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:57.341562 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.342145 (+ 583us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.342380 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 11:46:57.349581 32465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:46:57.367404 32465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:46:57.446028 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.434519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:36633 (local address 127.0.0.1:52066)
0808 11:46:57.437224 (+ 2705us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:57.437242 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:57.437368 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:57.438252 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:57.438256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:57.438281 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:57.438596 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.438608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.440628 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.440634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.443086 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.443106 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.444977 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.444984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.445326 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.445388 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 11:46:57.445492 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 11:46:57.445672 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2575,"thread_start_us":768,"threads_started":1}
I20260808 11:46:57.447412 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.435108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.13.158.66:36633)
0808 11:46:57.435678 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.435686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.437498 (+ 1812us) server_negotiation.cc:407] Connection header received
0808 11:46:57.437599 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.437606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.437819 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.438001 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:57.438855 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.440459 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.443284 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.444705 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.446585 (+ 1880us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.446814 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.446883 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.447085 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":346,"threads_started":1}
I20260808 11:46:57.451587 394 tablet_service.cc:1467] Processing CreateTablet for tablet 192956e009ea40b6b385a7c5465de3cc (DEFAULT_TABLE table=test-table [id=80c3a8eb016a49a8b6a2e22537bc95da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:46:57.453325 394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 192956e009ea40b6b385a7c5465de3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:46:57.458868 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.425686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:39305 (local address 127.0.0.1:38058)
0808 11:46:57.435068 (+ 9382us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:57.435085 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:57.435206 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:57.436654 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:57.436662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:57.436691 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:57.437159 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.437172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.442315 (+ 5143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.442321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.450993 (+ 8672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.451014 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.458056 (+ 7042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.458063 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.458236 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.458268 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:46:57.458353 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 11:46:57.458490 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9257,"thread_start_us":7205,"threads_started":1}
I20260808 11:46:57.459729 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.418970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55778 (local address 127.13.158.65:40579)
0808 11:46:57.431247 (+ 12277us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.431262 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.433548 (+ 2286us) server_negotiation.cc:407] Connection header received
0808 11:46:57.438123 (+ 4575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.438131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.438326 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.438468 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:57.445307 (+ 6839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.446880 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.452925 (+ 6045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.454358 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.454463 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.454670 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.459247 (+ 4577us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.459419 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12062,"thread_start_us":11889,"threads_started":1}
I20260808 11:46:57.460572 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.418534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:40579 (local address 127.0.0.1:55778)
0808 11:46:57.433309 (+ 14775us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:57.433369 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:57.433640 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:57.438757 (+ 5117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:57.438762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:57.438786 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:57.439128 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.439139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.448752 (+ 9613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.448756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.450549 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.450566 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.454986 (+ 4420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.454992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.455181 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.455213 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:46:57.460115 (+ 4902us) client_negotiation.cc:240] Negotiation successful
0808 11:46:57.460265 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14491,"thread_start_us":18764,"threads_started":1}
I20260808 11:46:57.464565 32660 tablet_service.cc:1467] Processing CreateTablet for tablet 192956e009ea40b6b385a7c5465de3cc (DEFAULT_TABLE table=test-table [id=80c3a8eb016a49a8b6a2e22537bc95da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:46:57.466327 32660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 192956e009ea40b6b385a7c5465de3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:46:57.476351 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.426400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.13.158.67:39305)
0808 11:46:57.434194 (+ 7794us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.434202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.435743 (+ 1541us) server_negotiation.cc:407] Connection header received
0808 11:46:57.435854 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.435862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.436122 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.436319 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:57.439336 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.442153 (+ 2817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.451183 (+ 9030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.456309 (+ 5126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.456435 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.475645 (+ 19210us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.475757 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.475962 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7625,"thread_start_us":9217,"threads_started":1}
I20260808 11:46:57.480481 589 tablet_service.cc:1467] Processing CreateTablet for tablet 192956e009ea40b6b385a7c5465de3cc (DEFAULT_TABLE table=test-table [id=80c3a8eb016a49a8b6a2e22537bc95da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:46:57.482223 589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 192956e009ea40b6b385a7c5465de3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:46:57.484937 717 tablet_bootstrap.cc:492] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Bootstrap starting.
I20260808 11:46:57.489639 717 tablet_bootstrap.cc:654] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Neither blocks nor log segments found. Creating new log.
I20260808 11:46:57.491281 717 log.cc:824] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:57.493835 719 tablet_bootstrap.cc:492] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Bootstrap starting.
I20260808 11:46:57.498574 719 tablet_bootstrap.cc:654] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Neither blocks nor log segments found. Creating new log.
I20260808 11:46:57.499883 719 log.cc:824] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:57.508239 717 tablet_bootstrap.cc:492] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: No bootstrap required, opened a new log
I20260808 11:46:57.508658 717 ts_tablet_manager.cc:1397] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.000s
I20260808 11:46:57.515786 723 tablet_bootstrap.cc:492] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Bootstrap starting.
I20260808 11:46:57.519560 723 tablet_bootstrap.cc:654] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Neither blocks nor log segments found. Creating new log.
I20260808 11:46:57.520881 723 log.cc:824] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:46:57.525151 719 tablet_bootstrap.cc:492] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: No bootstrap required, opened a new log
I20260808 11:46:57.525501 719 ts_tablet_manager.cc:1397] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.010s
I20260808 11:46:57.530814 723 tablet_bootstrap.cc:492] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: No bootstrap required, opened a new log
I20260808 11:46:57.531174 723 ts_tablet_manager.cc:1397] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260808 11:46:57.534319 717 raft_consensus.cc:348] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
I20260808 11:46:57.534881 717 raft_consensus.cc:374] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:46:57.535018 717 raft_consensus.cc:729] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e9ccdef60548068121bb3fa725dd40, State: Initialized, Role: FOLLOWER
I20260808 11:46:57.535802 717 consensus_queue.cc:260] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
I20260808 11:46:57.544382 723 raft_consensus.cc:348] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
I20260808 11:46:57.545009 723 raft_consensus.cc:374] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:46:57.545202 723 raft_consensus.cc:729] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392ceda019eb4e7ab957dfa2528ee04f, State: Initialized, Role: FOLLOWER
I20260808 11:46:57.546571 495 heartbeater.cc:503] Master 127.13.158.126:39607 was elected leader, sending a full tablet report...
I20260808 11:46:57.546757 723 consensus_queue.cc:260] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
I20260808 11:46:57.553961 692 heartbeater.cc:503] Master 127.13.158.126:39607 was elected leader, sending a full tablet report...
I20260808 11:46:57.554581 719 raft_consensus.cc:348] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
I20260808 11:46:57.555006 719 raft_consensus.cc:374] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:46:57.555051 723 ts_tablet_manager.cc:1428] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Time spent starting tablet: real 0.024s user 0.016s sys 0.006s
I20260808 11:46:57.555130 719 raft_consensus.cc:729] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56cc32be28ba4af6ab877f0db54b2187, State: Initialized, Role: FOLLOWER
I20260808 11:46:57.555713 719 consensus_queue.cc:260] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
I20260808 11:46:57.557224 717 ts_tablet_manager.cc:1428] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Time spent starting tablet: real 0.048s user 0.021s sys 0.004s
I20260808 11:46:57.559381 32748 heartbeater.cc:503] Master 127.13.158.126:39607 was elected leader, sending a full tablet report...
I20260808 11:46:57.559845 719 ts_tablet_manager.cc:1428] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Time spent starting tablet: real 0.034s user 0.015s sys 0.001s
W20260808 11:46:57.571018 32749 tablet.cc:2369] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:46:57.706198 694 tablet.cc:2369] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:46:57.727597 736 raft_consensus.cc:482] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:46:57.727964 736 raft_consensus.cc:504] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
I20260808 11:46:57.730149 736 leader_election.cc:288] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56cc32be28ba4af6ab877f0db54b2187 (127.13.158.65:40579), 392ceda019eb4e7ab957dfa2528ee04f (127.13.158.67:39305)
I20260808 11:46:57.739352 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.730819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:59047 (local address 127.13.158.65:40579)
0808 11:46:57.731053 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.731060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.731990 (+ 930us) server_negotiation.cc:407] Connection header received
0808 11:46:57.732167 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.732174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.732274 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.732383 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:57.733245 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.734107 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.736919 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.738123 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.738294 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.738368 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.738956 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.739087 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:46:57.739347 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.730618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:56643 (local address 127.13.158.67:39305)
0808 11:46:57.730847 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.730853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.731586 (+ 733us) server_negotiation.cc:407] Connection header received
0808 11:46:57.731917 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.731925 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.732027 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.732122 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:46:57.733236 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.734339 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.736916 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.738123 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.738294 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.738368 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.738956 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.739087 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 11:46:57.739573 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.730391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:39305 (local address 127.13.158.66:56643)
0808 11:46:57.731460 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:57.731481 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:57.731734 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:57.732444 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:57.732448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:57.732539 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:57.733002 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.733011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.734470 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.734474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.736752 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.736767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.738400 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.738408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.738749 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.738819 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 11:46:57.738912 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 11:46:57.739083 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":162,"spinlock_wait_cycles":22784,"thread_start_us":665,"threads_started":1}
I20260808 11:46:57.739620 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.730390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:40579 (local address 127.13.158.66:59047)
0808 11:46:57.731872 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0808 11:46:57.731889 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:46:57.732032 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:46:57.732448 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:46:57.732452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:46:57.732475 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:46:57.732990 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.733004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.734236 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.734244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.736753 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:46:57.736767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.738417 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.738422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:46:57.738752 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.738813 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 11:46:57.738912 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 11:46:57.739083 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1350,"mutex_wait_us":720,"thread_start_us":592,"threads_started":1}
I20260808 11:46:57.740658 624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192956e009ea40b6b385a7c5465de3cc" candidate_uuid: "99e9ccdef60548068121bb3fa725dd40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "392ceda019eb4e7ab957dfa2528ee04f" is_pre_election: true
I20260808 11:46:57.740785 32688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192956e009ea40b6b385a7c5465de3cc" candidate_uuid: "99e9ccdef60548068121bb3fa725dd40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56cc32be28ba4af6ab877f0db54b2187" is_pre_election: true
I20260808 11:46:57.741127 624 raft_consensus.cc:2454] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99e9ccdef60548068121bb3fa725dd40 in term 0.
I20260808 11:46:57.741345 32688 raft_consensus.cc:2454] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99e9ccdef60548068121bb3fa725dd40 in term 0.
I20260808 11:46:57.741791 330 leader_election.cc:302] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 392ceda019eb4e7ab957dfa2528ee04f, 99e9ccdef60548068121bb3fa725dd40; no voters:
I20260808 11:46:57.742235 736 raft_consensus.cc:2781] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:46:57.742422 736 raft_consensus.cc:482] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:46:57.742535 736 raft_consensus.cc:3037] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:46:57.744788 736 raft_consensus.cc:504] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
I20260808 11:46:57.745502 736 leader_election.cc:288] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [CANDIDATE]: Term 1 election: Requested vote from peers 56cc32be28ba4af6ab877f0db54b2187 (127.13.158.65:40579), 392ceda019eb4e7ab957dfa2528ee04f (127.13.158.67:39305)
I20260808 11:46:57.745970 32688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192956e009ea40b6b385a7c5465de3cc" candidate_uuid: "99e9ccdef60548068121bb3fa725dd40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56cc32be28ba4af6ab877f0db54b2187"
I20260808 11:46:57.746059 624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192956e009ea40b6b385a7c5465de3cc" candidate_uuid: "99e9ccdef60548068121bb3fa725dd40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "392ceda019eb4e7ab957dfa2528ee04f"
I20260808 11:46:57.746214 32688 raft_consensus.cc:3037] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:46:57.746330 624 raft_consensus.cc:3037] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:46:57.748932 32688 raft_consensus.cc:2454] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e9ccdef60548068121bb3fa725dd40 in term 1.
I20260808 11:46:57.749370 329 leader_election.cc:302] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56cc32be28ba4af6ab877f0db54b2187, 99e9ccdef60548068121bb3fa725dd40; no voters:
I20260808 11:46:57.749693 736 raft_consensus.cc:2781] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:46:57.749785 624 raft_consensus.cc:2454] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e9ccdef60548068121bb3fa725dd40 in term 1.
I20260808 11:46:57.750277 736 raft_consensus.cc:686] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 1 LEADER]: Becoming Leader. State: Replica: 99e9ccdef60548068121bb3fa725dd40, State: Running, Role: LEADER
I20260808 11:46:57.751232 736 consensus_queue.cc:237] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } }
W20260808 11:46:57.751850 496 tablet.cc:2369] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:46:57.756263 32465 catalog_manager.cc:5495] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99e9ccdef60548068121bb3fa725dd40 (127.13.158.66). New cstate: current_term: 1 leader_uuid: "99e9ccdef60548068121bb3fa725dd40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e9ccdef60548068121bb3fa725dd40" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 36633 } health_report { overall_health: HEALTHY } } }
I20260808 11:46:57.834039 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.828605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52080 (local address 127.13.158.66:36633)
0808 11:46:57.828826 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.828832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.828853 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:46:57.828963 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.828967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.829055 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.829221 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:46:57.829595 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.830329 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.831800 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.832228 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.832366 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.832956 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:46:57.833448 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:46:57.833603 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.833684 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.833785 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:46:57.841850 32688 raft_consensus.cc:1261] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 1 FOLLOWER]: Refusing update from remote peer 99e9ccdef60548068121bb3fa725dd40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:46:57.841862 624 raft_consensus.cc:1261] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 1 FOLLOWER]: Refusing update from remote peer 99e9ccdef60548068121bb3fa725dd40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:46:57.842906 736 consensus_queue.cc:1035] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "392ceda019eb4e7ab957dfa2528ee04f" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 39305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:46:57.843254 744 consensus_queue.cc:1035] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56cc32be28ba4af6ab877f0db54b2187" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:46:57.871266 32465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:46:57.871551 32465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:46:57.874939 32465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ca8a61029a34e74a09687a7cecdd934: Sending DeleteTablet for 3 replicas of tablet 192956e009ea40b6b385a7c5465de3cc
I20260808 11:46:57.876056 32660 tablet_service.cc:1514] Processing DeleteTablet for tablet 192956e009ea40b6b385a7c5465de3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:46:57 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:55778
I20260808 11:46:57.876228 589 tablet_service.cc:1514] Processing DeleteTablet for tablet 192956e009ea40b6b385a7c5465de3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:46:57 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:38058
I20260808 11:46:57.876873 394 tablet_service.cc:1514] Processing DeleteTablet for tablet 192956e009ea40b6b385a7c5465de3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:46:57 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:52066
I20260808 11:46:57.877033 766 tablet_replica.cc:330] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: stopping tablet replica
I20260808 11:46:57.877455 766 raft_consensus.cc:2229] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:46:57.877954 766 raft_consensus.cc:2258] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:46:57.880324 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 32515
I20260808 11:46:57.880348 766 ts_tablet_manager.cc:1905] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:46:57.888872 767 tablet_replica.cc:330] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: stopping tablet replica
I20260808 11:46:57.889406 767 raft_consensus.cc:2229] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:46:57.890129 767 raft_consensus.cc:2258] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:46:57.891322 768 tablet_replica.cc:330] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: stopping tablet replica
I20260808 11:46:57.891816 768 raft_consensus.cc:2229] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:46:57.892573 768 raft_consensus.cc:2258] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:46:57.892737 767 ts_tablet_manager.cc:1905] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:46:57.894611 768 ts_tablet_manager.cc:1905] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:46:57.897698 766 ts_tablet_manager.cc:1918] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:46:57.899868 767 ts_tablet_manager.cc:1918] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:46:57.900089 767 log.cc:1196] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/192956e009ea40b6b385a7c5465de3cc
I20260808 11:46:57.900694 767 ts_tablet_manager.cc:1939] T 192956e009ea40b6b385a7c5465de3cc P 392ceda019eb4e7ab957dfa2528ee04f: Deleting consensus metadata
I20260808 11:46:57.902025 768 ts_tablet_manager.cc:1918] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:46:57.902259 768 log.cc:1196] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/192956e009ea40b6b385a7c5465de3cc
I20260808 11:46:57.902521 766 log.cc:1196] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/192956e009ea40b6b385a7c5465de3cc
I20260808 11:46:57.902679 32443 catalog_manager.cc:4841] TS 392ceda019eb4e7ab957dfa2528ee04f (127.13.158.67:39305): tablet 192956e009ea40b6b385a7c5465de3cc (table test-table [id=80c3a8eb016a49a8b6a2e22537bc95da]) successfully deleted
I20260808 11:46:57.903015 768 ts_tablet_manager.cc:1939] T 192956e009ea40b6b385a7c5465de3cc P 99e9ccdef60548068121bb3fa725dd40: Deleting consensus metadata
I20260808 11:46:57.903435 766 ts_tablet_manager.cc:1939] T 192956e009ea40b6b385a7c5465de3cc P 56cc32be28ba4af6ab877f0db54b2187: Deleting consensus metadata
I20260808 11:46:57.904570 32441 catalog_manager.cc:4841] TS 99e9ccdef60548068121bb3fa725dd40 (127.13.158.66:36633): tablet 192956e009ea40b6b385a7c5465de3cc (table test-table [id=80c3a8eb016a49a8b6a2e22537bc95da]) successfully deleted
I20260808 11:46:57.905213 32442 catalog_manager.cc:4841] TS 56cc32be28ba4af6ab877f0db54b2187 (127.13.158.65:40579): tablet 192956e009ea40b6b385a7c5465de3cc (table test-table [id=80c3a8eb016a49a8b6a2e22537bc95da]) successfully deleted
I20260808 11:46:57.918094 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:57.883138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55790 (local address 127.13.158.65:40579)
0808 11:46:57.883371 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:57.883380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:57.900636 (+ 17256us) server_negotiation.cc:407] Connection header received
0808 11:46:57.900880 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:57.900885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:57.900969 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:57.901064 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:57.905415 (+ 4351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.906554 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.907895 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:57.908536 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:57.908703 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:57.911253 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:57.911269 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:57.911287 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:57.911335 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:57.915231 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:57.915906 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:57.915916 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:57.915921 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:57.916087 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:57.916489 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:57.916495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:57.916498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:57.916953 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:57.917278 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:57.917438 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 11:46:57.917693 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:46:57.918887 32742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:58.032521 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 32515
I20260808 11:46:58.061659 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 32753
I20260808 11:46:58.087404 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:58.062466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.13.158.66:36633)
0808 11:46:58.062632 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:58.062637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:58.062882 (+ 245us) server_negotiation.cc:407] Connection header received
0808 11:46:58.063096 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:58.063102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:58.063178 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:58.063273 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:58.065846 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:58.067477 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:58.070525 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:58.071481 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:58.071606 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:58.077790 (+ 6184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:58.077809 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:58.077831 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:58.077934 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:58.083190 (+ 5256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:58.084391 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:58.084398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:58.084400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:58.084648 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:58.085275 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:58.085278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:58.085280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:58.085764 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:58.086243 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:58.086561 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 11:46:58.086806 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:46:58.088320 486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:58.217770 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 32753
I20260808 11:46:58.248152 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 502
I20260808 11:46:58.261513 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:46:58.248937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38060 (local address 127.13.158.67:39305)
0808 11:46:58.249420 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 11:46:58.249425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:46:58.249442 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:46:58.249497 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:46:58.249500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:46:58.249561 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:46:58.249654 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:46:58.251018 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:58.251876 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:58.253186 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:46:58.253583 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:46:58.253775 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:46:58.256200 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:46:58.256213 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:46:58.256229 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:46:58.256277 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:46:58.259045 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:58.259675 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:58.259682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:58.259684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:58.259814 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:46:58.260129 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:46:58.260141 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:46:58.260143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:46:58.260484 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:46:58.260703 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 11:46:58.260951 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 11:46:58.261106 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":272,"threads_started":1}
I20260808 11:46:58.262171 682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:58.353314 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 502
I20260808 11:46:58.379169 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 32418
I20260808 11:46:58.380343 32503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:46:58.505362 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 32418
I20260808 11:46:58.540760 13945 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 11:46:58.886233 13945 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 11:46:58.886444 13945 mini_ranger.cc:66] Stopping Ranger...
I20260808 11:46:59.242913 13945 mini_ranger.cc:68] Stopped Ranger
2026-08-08 11:46:59.243 UTC [31019] LOG: received smart shutdown request
2026-08-08 11:46:59.246 UTC [31019] LOG: background worker "logical replication launcher" (PID 31036) exited with exit code 1
2026-08-08 11:46:59.246 UTC [31026] LOG: shutting down
2026-08-08 11:46:59.395 UTC [31019] LOG: database system is shut down
2026-08-08T11:46:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53319 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:46:59 dist-test-slave-39l8 krb5kdc[796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:46:59 dist-test-slave-39l8 krb5kdc[796](info): set up 2 sockets
Aug 08 11:46:59 dist-test-slave-39l8 krb5kdc[796](info): commencing operation
krb5kdc: starting...
W20260808 11:47:00.847572 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:47:01 dist-test-slave-39l8 krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:47:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:47:01.118440 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:38303
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:38303,127.13.158.125:44583,127.13.158.124:45943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:01.458511 815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:01.458860 815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:01.459064 815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:01.466501 815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:01.466609 815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:01.466701 815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:01.466825 815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:01.466918 815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:01.477094 815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:38303,127.13.158.125:44583,127.13.158.124:45943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:38303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:01.478451 815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:01.479801 815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:01.492067 823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:01.492161 826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:01.492280 824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:01.493513 815 server_base.cc:1034] running on GCE node
I20260808 11:47:01.494282 815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:01.495725 815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:01.496956 815 hybrid_clock.cc:648] HybridClock initialized: now 1786189621496922 us; error 47 us; skew 500 ppm
I20260808 11:47:01.497299 815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:01 dist-test-slave-39l8 krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189621, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:01.500316 815 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:47:01.501796 815 webserver.cc:460] Webserver started at http://127.13.158.126:44391/ using document root <none> and password file <none>
I20260808 11:47:01.502521 815 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:01.502664 815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:01.502997 815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:01.505892 815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1"
format_stamp: "Formatted at 2026-08-08 11:47:01 on dist-test-slave-39l8"
server_key: "0cf9ed6c1f76b4996cb33830ffce3e05"
server_key_iv: "c09d8ee1f10587a2b08bdc3a0bf76ddc"
server_key_version: "encryptionkey@0"
I20260808 11:47:01.506748 815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1"
format_stamp: "Formatted at 2026-08-08 11:47:01 on dist-test-slave-39l8"
server_key: "0cf9ed6c1f76b4996cb33830ffce3e05"
server_key_iv: "c09d8ee1f10587a2b08bdc3a0bf76ddc"
server_key_version: "encryptionkey@0"
I20260808 11:47:01.511739 815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 11:47:01.515192 834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:01.516469 815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:47:01.516659 815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1"
format_stamp: "Formatted at 2026-08-08 11:47:01 on dist-test-slave-39l8"
server_key: "0cf9ed6c1f76b4996cb33830ffce3e05"
server_key_iv: "c09d8ee1f10587a2b08bdc3a0bf76ddc"
server_key_version: "encryptionkey@0"
I20260808 11:47:01.516851 815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:01.560649 815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:01.564729 815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:01.565049 815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:01.583694 815 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:38303
I20260808 11:47:01.583719 908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:38303 every 8 connection(s)
I20260808 11:47:01.585072 815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:47:01.585758 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 815
I20260808 11:47:01.586032 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:47:01.586853 13945 external_mini_cluster.cc:1428] Setting key 26d3c746355c9eb34699121ad5e4142f
I20260808 11:47:01.588989 909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:01.595464 909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:47:01 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189621, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.158.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.125@KRBTEST.COM" created.
I20260808 11:47:01.619946 921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.599388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53030 (local address 127.13.158.126:38303)
0808 11:47:01.599993 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:01.599999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:01.600039 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:47:01.602393 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:01.602398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:01.602604 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:01.602833 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:01.604494 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:01.605670 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:01.607999 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:01.608587 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:01.608731 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:01.612791 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:01.612800 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:01.612812 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:01.612846 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:01.615622 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:01.616313 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:01.616319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:01.616322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:01.616505 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:01.616903 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:01.616908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:01.616911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:01.617394 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:01.617608 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:01.618140 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 11:47:01.618461 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":289,"threads_started":1}
I20260808 11:47:01.620015 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.598369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:38303 (local address 127.0.0.1:53030)
0808 11:47:01.599178 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.599234 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:01.602011 (+ 2777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:01.602868 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:01.602878 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:01.603766 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:01.604287 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:01.604316 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:01.605848 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:01.605857 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:01.607836 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:01.607855 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:01.608847 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:01.608859 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:01.609097 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:01.609835 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:01.609884 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:01.612563 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:01.615765 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:01.615785 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:01.615797 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:01.616170 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:01.616615 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:01.616621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:01.616625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:01.616787 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:01.617532 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:01.617545 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:01.617756 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 11:47:01.617995 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 11:47:01.618598 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":139,"thread_start_us":319,"threads_started":1}
I20260808 11:47:01.623814 909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:47:01.624950 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.624553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60220)
0808 11:47:01.624731 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.624840 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 11:47:01.625240 837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
I20260808 11:47:01.627215 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.626677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60224)
0808 11:47:01.626916 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.627011 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260808 11:47:01.627444 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:44583: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab.
Entry for principal kudu/127.13.158.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab.
I20260808 11:47:01.649791 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.125:44583
--webserver_interface=127.13.158.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab
--principal=kudu/127.13.158.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:38303,127.13.158.125:44583,127.13.158.124:45943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:47:01.676787 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } attempt: 1
I20260808 11:47:01.678041 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.677568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60228)
0808 11:47:01.677772 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.677906 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 11:47:01.679126 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.678744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60232)
0808 11:47:01.678891 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.678968 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 11:47:01.679335 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:44583: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
I20260808 11:47:01.747653 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } attempt: 2
I20260808 11:47:01.748888 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.748395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60234)
0808 11:47:01.748656 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.748751 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 11:47:01.749970 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.749626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60242)
0808 11:47:01.749776 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.749841 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:47:01.750270 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:44583: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
I20260808 11:47:01.841569 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } attempt: 3
I20260808 11:47:01.842962 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.842384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60250)
0808 11:47:01.842660 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.842828 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 11:47:01.843971 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.843598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60254)
0808 11:47:01.843765 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.843844 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 11:47:01.844172 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:44583: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
I20260808 11:47:01.987422 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } attempt: 4
W20260808 11:47:01.987461 924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:01.987879 924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:01.988041 924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 11:47:01.988590 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.988076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60256)
0808 11:47:01.988371 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.988459 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260808 11:47:01.989545 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:01.989198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60260)
0808 11:47:01.989341 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:01.989430 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 11:47:01.989730 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:44583: Network error: Client connection negotiation failed: client connection to 127.13.158.125:44583: connect: Connection refused (error 111)
W20260808 11:47:01.995447 924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:01.995615 924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:01.995716 924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:01.995832 924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:01.995910 924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:02.005868 924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:38303,127.13.158.125:44583,127.13.158.124:45943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab
--principal=kudu/127.13.158.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.125:44583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.158.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:02.007592 924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:02.008982 924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:02.023908 932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:02.024494 931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:02.024588 934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:02.024986 924 server_base.cc:1034] running on GCE node
I20260808 11:47:02.025870 924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:02.027375 924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:02.028610 924 hybrid_clock.cc:648] HybridClock initialized: now 1786189622028570 us; error 55 us; skew 500 ppm
I20260808 11:47:02.028975 924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189622, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:02.032266 924 init.cc:377] Logged in from keytab as kudu/127.13.158.125@KRBTEST.COM (short username kudu)
I20260808 11:47:02.033756 924 webserver.cc:460] Webserver started at http://127.13.158.125:38747/ using document root <none> and password file <none>
I20260808 11:47:02.034479 924 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:02.034627 924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:02.034974 924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:02.037995 924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/data/instance:
uuid: "d61fb27c1c074b29846a7cc18c7123f4"
format_stamp: "Formatted at 2026-08-08 11:47:02 on dist-test-slave-39l8"
server_key: "b85b2d74ea976d9f155a1369660d97cd"
server_key_iv: "dc86f5448bfc7dd736da5206db61cb81"
server_key_version: "encryptionkey@0"
I20260808 11:47:02.038921 924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/wal/instance:
uuid: "d61fb27c1c074b29846a7cc18c7123f4"
format_stamp: "Formatted at 2026-08-08 11:47:02 on dist-test-slave-39l8"
server_key: "b85b2d74ea976d9f155a1369660d97cd"
server_key_iv: "dc86f5448bfc7dd736da5206db61cb81"
server_key_version: "encryptionkey@0"
I20260808 11:47:02.044200 924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 11:47:02.047705 943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:02.049012 924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:47:02.049218 924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/wal
uuid: "d61fb27c1c074b29846a7cc18c7123f4"
format_stamp: "Formatted at 2026-08-08 11:47:02 on dist-test-slave-39l8"
server_key: "b85b2d74ea976d9f155a1369660d97cd"
server_key_iv: "dc86f5448bfc7dd736da5206db61cb81"
server_key_version: "encryptionkey@0"
I20260808 11:47:02.049427 924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:02.066694 924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:02.071297 924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:02.071621 924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:02.090138 924 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.125:44583
I20260808 11:47:02.090179 1009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.125:44583 every 8 connection(s)
I20260808 11:47:02.091678 924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
I20260808 11:47:02.095335 1012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:02.097575 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 924
I20260808 11:47:02.097788 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-1/wal/instance
I20260808 11:47:02.098264 13945 external_mini_cluster.cc:1428] Setting key 9271075ec0bd47b53f7039434c27bde7
I20260808 11:47:02.101981 1012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189622, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:02.125824 921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.104715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.13.158.126:38303)
0808 11:47:02.104944 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.104951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.106206 (+ 1255us) server_negotiation.cc:407] Connection header received
0808 11:47:02.109230 (+ 3024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.109239 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.109356 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.109481 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.111047 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.112269 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.113754 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.114506 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.114662 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.118433 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.118442 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.118449 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.118496 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.122477 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.123268 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.123276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.123282 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.123441 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.123851 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.123859 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.123865 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.124259 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.124421 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.125154 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.125409 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
WARNING: no policy specified for kudu/127.13.158.124@KRBTEST.COM; defaulting to no policy
I20260808 11:47:02.126606 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.105166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:38303 (local address 127.0.0.1:53034)
0808 11:47:02.106023 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.106076 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:02.108933 (+ 2857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:02.109686 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:02.109694 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:02.110225 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:02.110837 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.110875 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.112423 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.112427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.113599 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.113615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.114793 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.114800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.114979 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.115718 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:02.115742 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:02.118161 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:02.122620 (+ 4459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.122627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.122641 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.123127 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.123563 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.123566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.123568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.123728 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.124385 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:02.124392 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:02.124734 (+ 342us) client_negotiation.cc:769] Sending connection context
0808 11:47:02.124977 (+ 243us) client_negotiation.cc:240] Negotiation successful
0808 11:47:02.125358 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":320,"threads_started":1}
I20260808 11:47:02.128394 1012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.124@KRBTEST.COM" created.
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189622, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:47:02.143846 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.129276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60270)
0808 11:47:02.129458 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.129478 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:02.129575 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:02.130547 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:02.130551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:02.131006 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:02.131147 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.131154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.132490 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.132497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.133317 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.133327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.134124 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.134129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.134342 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.134911 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:02.134928 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:02.137272 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:02.141463 (+ 4191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.141467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.141469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.141858 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.142258 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.142261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.142263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.142396 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.143159 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:02.143162 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:02.143268 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 11:47:02.143377 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 11:47:02.143588 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 11:47:02.144352 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.129464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.13.158.125:44583)
0808 11:47:02.130163 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.130168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.130193 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:47:02.130241 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.130247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.130407 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.130554 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.131300 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.132358 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.133451 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.133975 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.134103 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.137473 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.137483 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.137495 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.137523 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.141325 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.141988 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.141999 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.142005 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.142147 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.142511 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.142519 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.142525 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.143036 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.143239 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.143561 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.143855 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":422,"threads_started":1}
I20260808 11:47:02.146798 1012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:47:02.148038 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.147492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38158)
0808 11:47:02.147703 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.147851 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 11:47:02.148344 949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
I20260808 11:47:02.149966 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.149540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38166)
0808 11:47:02.149734 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.149833 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 11:47:02.150221 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab.
Entry for principal kudu/127.13.158.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab.
I20260808 11:47:02.163295 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.124:45943
--webserver_interface=127.13.158.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab
--principal=kudu/127.13.158.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:38303,127.13.158.125:44583,127.13.158.124:45943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:47:02.199568 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 1
I20260808 11:47:02.200814 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.200280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38180)
0808 11:47:02.200500 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.200620 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 11:47:02.201781 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.201407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38188)
0808 11:47:02.201570 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.201653 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:47:02.202000 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
I20260808 11:47:02.270269 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 2
I20260808 11:47:02.271572 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.271020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38194)
0808 11:47:02.271270 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.271381 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 11:47:02.272754 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.272276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38206)
0808 11:47:02.272457 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.272560 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 11:47:02.272997 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
I20260808 11:47:02.289007 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } attempt: 5
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189621, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:47:02.303828 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.289869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.13.158.125:44583)
0808 11:47:02.290034 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.290039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.290058 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:47:02.290241 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.290244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.290333 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.290443 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.291456 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.292169 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.293821 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.294234 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.294328 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.298412 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.298417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.298420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.298452 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.301141 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.302023 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.302027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.302029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.302139 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.302533 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.302536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.302538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.302884 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.303007 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.303436 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.303602 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:47:02.304000 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.289743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60282)
0808 11:47:02.289957 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.289979 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:02.290110 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:02.290512 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:02.290517 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:02.291037 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:02.291298 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.291311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.292303 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.292308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.293653 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.293669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.294381 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.294388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.294600 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.295296 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:02.295324 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:02.298199 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:02.301308 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.301314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.301318 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.301908 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.302258 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.302263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.302266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.302432 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.303011 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:02.303017 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:02.303197 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 11:47:02.303357 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 11:47:02.303672 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 11:47:02.305190 909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:47:02.306433 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.305927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38214)
0808 11:47:02.306140 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.306249 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 11:47:02.307448 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.307092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38228)
0808 11:47:02.307249 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.307328 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 11:47:02.307664 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
I20260808 11:47:02.358942 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 1
I20260808 11:47:02.360421 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.359730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38234)
0808 11:47:02.359995 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.360172 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 11:47:02.361660 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.361192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38236)
0808 11:47:02.361377 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.361480 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 11:47:02.361967 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
I20260808 11:47:02.364239 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 3
I20260808 11:47:02.365388 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.364841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38250)
0808 11:47:02.365061 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.365185 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 11:47:02.366657 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.366104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38254)
0808 11:47:02.366383 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.366496 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260808 11:47:02.366971 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
I20260808 11:47:02.430222 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 2
I20260808 11:47:02.431607 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.431022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38258)
0808 11:47:02.431251 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.431362 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 11:47:02.432775 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.432374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38260)
0808 11:47:02.432527 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.432632 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:47:02.433013 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
W20260808 11:47:02.508512 1021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:02.508803 1021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:02.508985 1021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 11:47:02.510211 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 4
I20260808 11:47:02.511451 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.510909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38276)
0808 11:47:02.511148 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.511258 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 11:47:02.512491 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.512129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38288)
0808 11:47:02.512285 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.512368 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 11:47:02.512730 1012 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
W20260808 11:47:02.517134 1021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:02.517246 1021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:02.517326 1021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:02.517427 1021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:02.517488 1021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:02.526810 1021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:38303,127.13.158.125:44583,127.13.158.124:45943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab
--principal=kudu/127.13.158.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.124:45943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.158.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:02.528404 1021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:02.529721 1021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 11:47:02.539265 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 3
I20260808 11:47:02.540606 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.539923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38296)
0808 11:47:02.540283 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.540392 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260808 11:47:02.541201 1036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:02.541296 1040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:02.541886 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.541356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38306)
0808 11:47:02.541668 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.541759 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
W20260808 11:47:02.541939 1031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:02.542120 909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:45943: Network error: Client connection negotiation failed: client connection to 127.13.158.124:45943: connect: Connection refused (error 111)
I20260808 11:47:02.543213 1021 server_base.cc:1034] running on GCE node
I20260808 11:47:02.544054 1021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:02.545502 1021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:02.546777 1021 hybrid_clock.cc:648] HybridClock initialized: now 1786189622546713 us; error 56 us; skew 500 ppm
I20260808 11:47:02.547200 1021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189622, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:02.550194 1021 init.cc:377] Logged in from keytab as kudu/127.13.158.124@KRBTEST.COM (short username kudu)
I20260808 11:47:02.551749 1021 webserver.cc:460] Webserver started at http://127.13.158.124:37081/ using document root <none> and password file <none>
I20260808 11:47:02.552423 1021 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:02.552522 1021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:02.552928 1021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:02.556102 1021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/data/instance:
uuid: "97570e8105b84f969c245e1f8e7b11a8"
format_stamp: "Formatted at 2026-08-08 11:47:02 on dist-test-slave-39l8"
server_key: "66aca140f05b77a5f274a184a8ec7b10"
server_key_iv: "a9ed29f8eb68faf784aafb74672fe4af"
server_key_version: "encryptionkey@0"
I20260808 11:47:02.556927 1021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/wal/instance:
uuid: "97570e8105b84f969c245e1f8e7b11a8"
format_stamp: "Formatted at 2026-08-08 11:47:02 on dist-test-slave-39l8"
server_key: "66aca140f05b77a5f274a184a8ec7b10"
server_key_iv: "a9ed29f8eb68faf784aafb74672fe4af"
server_key_version: "encryptionkey@0"
I20260808 11:47:02.562323 1021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:47:02.565949 1050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:02.567324 1021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:02.567507 1021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/wal
uuid: "97570e8105b84f969c245e1f8e7b11a8"
format_stamp: "Formatted at 2026-08-08 11:47:02 on dist-test-slave-39l8"
server_key: "66aca140f05b77a5f274a184a8ec7b10"
server_key_iv: "a9ed29f8eb68faf784aafb74672fe4af"
server_key_version: "encryptionkey@0"
I20260808 11:47:02.567754 1021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:02.615123 1021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:02.619443 1021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:02.619735 1021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:02.640455 1021 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.124:45943
I20260808 11:47:02.640486 1124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.124:45943 every 8 connection(s)
I20260808 11:47:02.641876 1021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
I20260808 11:47:02.645727 1125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:02.649745 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 1021
I20260808 11:47:02.649947 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189414476033-13945-0/minicluster-data/master-2/wal/instance
I20260808 11:47:02.650424 13945 external_mini_cluster.cc:1428] Setting key 4c868b6ada715d8fd85e8bae82c6513a
I20260808 11:47:02.653049 1125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189622, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
I20260808 11:47:02.674170 1129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.657570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53046 (local address 127.13.158.126:38303)
0808 11:47:02.658116 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.658122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.658733 (+ 611us) server_negotiation.cc:407] Connection header received
0808 11:47:02.658970 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.658976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.659056 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.659199 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.660478 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.661165 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.662869 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.663458 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.663551 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.667661 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.667666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.667669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.667703 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.670537 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.671518 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.671525 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.671529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.671678 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.672161 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.672166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.672169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.672517 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.672629 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.673507 (+ 878us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.673750 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":269,"threads_started":1}
I20260808 11:47:02.675148 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.652441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.13.158.124:45943)
0808 11:47:02.653416 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.653432 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.653484 (+ 52us) server_negotiation.cc:407] Connection header received
0808 11:47:02.655433 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.655472 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.655973 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.656413 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.658099 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.659884 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.661572 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.662285 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.662430 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.668299 (+ 5869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.668307 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.668318 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.668344 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.671196 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.672104 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.672113 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.672115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.672284 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.672832 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.672835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.672837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.673196 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.673481 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.673822 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.674051 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":698,"thread_start_us":415,"threads_started":1}
I20260808 11:47:02.675163 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.657534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:38303 (local address 127.0.0.1:53046)
0808 11:47:02.658527 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.658580 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:02.658811 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:02.659261 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:02.659266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:02.659907 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:02.660317 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.660329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.661355 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.661360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.662678 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.662695 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.663721 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.663729 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.663929 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.664529 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:02.664557 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:02.667453 (+ 2896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:02.670691 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.670728 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.670740 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.671382 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.671801 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.671803 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.671806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.671961 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.672645 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:02.672654 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:02.673072 (+ 418us) client_negotiation.cc:769] Sending connection context
0808 11:47:02.673368 (+ 296us) client_negotiation.cc:240] Negotiation successful
0808 11:47:02.673881 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":2,"thread_start_us":545,"threads_started":1}
I20260808 11:47:02.677006 1125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189622, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:02.691478 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.677795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.13.158.125:44583)
0808 11:47:02.677986 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.677991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.678012 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:47:02.678075 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.678080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.678163 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.678267 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.679164 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.680271 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.681902 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.682593 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.682750 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.685900 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.685905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.685908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.685941 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.688741 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.689567 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.689570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.689572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.689738 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.690204 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.690208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.690210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.690443 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.690552 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.691029 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.691202 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:47:02.691641 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.677689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:44583 (local address 127.0.0.1:60296)
0808 11:47:02.677860 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.677879 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:02.677975 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:02.678302 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:02.678307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:02.678827 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:02.679018 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.679033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.680421 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.680425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.681734 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.681749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.682788 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.682792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.682947 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.683538 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:02.683555 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:02.685705 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:02.688897 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.688915 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.688920 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.689444 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.689878 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.689883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.689887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.690058 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.690561 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:02.690566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:02.690768 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 11:47:02.690943 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 11:47:02.691271 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 11:47:02.692776 1125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:47:02.694342 1129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.678647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53054 (local address 127.13.158.126:38303)
0808 11:47:02.678832 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.678842 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.678903 (+ 61us) server_negotiation.cc:407] Connection header received
0808 11:47:02.679065 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.679068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.679119 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.679191 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.680686 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.681687 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.683413 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.683834 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.683967 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.688045 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.688050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.688053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.688106 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.691473 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.692320 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.692325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.692329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.692457 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.692974 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.692979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.692983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.693280 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.693552 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.693846 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.694070 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189622, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:47:02.707590 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.693721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38314)
0808 11:47:02.693953 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.693965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:02.694084 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:02.694630 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:02.694634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:02.695097 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:02.695283 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.695295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.696367 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.696372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.697227 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.697237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.697888 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.697893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.698121 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.698693 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:02.698758 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:02.700878 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:02.704294 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.704301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.704304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.704810 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.705251 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.705276 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.705281 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.705465 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.706478 (+ 1013us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:02.706482 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:02.706696 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 11:47:02.706897 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 11:47:02.707195 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260808 11:47:02.708120 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.693687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.13.158.124:45943)
0808 11:47:02.693861 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.693867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.694032 (+ 165us) server_negotiation.cc:407] Connection header received
0808 11:47:02.694327 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.694330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.694378 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.694490 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.695429 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.696248 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.697375 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.697750 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.697868 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.701084 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.701088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.701092 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.701120 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.704144 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.704953 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.704958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.704962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.705135 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.705830 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.705836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.705839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.706252 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.706433 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.707045 (+ 612us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.707394 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":68}
I20260808 11:47:02.711256 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.696097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.13.158.125:44583)
0808 11:47:02.696247 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.696253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.696271 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:47:02.696422 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.696428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.696495 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.696596 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.697718 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.698436 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.699790 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.700211 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.700334 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.704907 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.704915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.704920 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.704973 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.708357 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.709299 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.709304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.709307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.709452 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.709885 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.709892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.709898 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.710181 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.710443 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.710622 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.710921 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:47:02.716089 1125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8: Bootstrap starting.
I20260808 11:47:02.716225 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:47:02.719396 909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 4
I20260808 11:47:02.719504 1125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:02.720925 1125 log.cc:824] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189621, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
I20260808 11:47:02.731732 1125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8: No bootstrap required, opened a new log
I20260808 11:47:02.747746 1125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.748250 1125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:02.748396 1125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97570e8105b84f969c245e1f8e7b11a8, State: Initialized, Role: FOLLOWER
I20260808 11:47:02.749197 1125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.749727 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.720019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38328)
0808 11:47:02.720364 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.720385 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:02.720496 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:02.721598 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:02.721603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:02.722193 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:02.722487 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.722503 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.724474 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.724482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.726393 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.726413 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.727806 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.727812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.728018 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.728868 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:02.728901 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:02.732271 (+ 3370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:02.746090 (+ 13819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.746099 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.746107 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.746879 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.747459 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.747466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.747472 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.747667 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.748484 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:02.748491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:02.748689 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 11:47:02.748872 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 11:47:02.749255 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260808 11:47:02.751204 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.720378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.13.158.124:45943)
0808 11:47:02.721253 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.721259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.721277 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:47:02.721329 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.721333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.721408 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.722673 (+ 1265us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.722876 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.724266 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.726634 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.727610 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.732480 (+ 4870us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.732759 (+ 279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.732767 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.732779 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.732817 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.741583 (+ 8766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.747089 (+ 5506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.747096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.747103 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.747303 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.747837 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.747845 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.747851 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.748318 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.750170 (+ 1852us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.750336 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.750641 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739}
I20260808 11:47:02.757438 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.721808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.13.158.125:44583)
0808 11:47:02.721980 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.721985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.729219 (+ 7234us) server_negotiation.cc:407] Connection header received
0808 11:47:02.739113 (+ 9894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.739123 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.739228 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.739312 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.741569 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.742674 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.744886 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.745551 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.745650 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.749433 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.749441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.749447 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.749486 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.753668 (+ 4182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.754829 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.754835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.754839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.754996 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.755557 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.755563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.755566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.755969 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.756213 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.756863 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.757107 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:47:02.757402 909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: Bootstrap starting.
I20260808 11:47:02.761798 909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:02.763332 909 log.cc:824] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:02.764530 1129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.721269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.13.158.126:38303)
0808 11:47:02.721464 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.721472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.735320 (+ 13848us) server_negotiation.cc:407] Connection header received
0808 11:47:02.735613 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.735618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.735691 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.735758 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.741802 (+ 6044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.743234 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.752465 (+ 9231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.753215 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.753337 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.756998 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.757006 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.757011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.757063 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.760944 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.761965 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.761970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.761973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.762136 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.762625 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.762630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.762633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.763088 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.763373 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.763809 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.764068 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:47:02.767726 909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: No bootstrap required, opened a new log
I20260808 11:47:02.768801 1145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.769302 1145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:47:02.769665 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.727801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38342 (local address 127.13.158.124:45943)
0808 11:47:02.740517 (+ 12716us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.740524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.752140 (+ 11616us) server_negotiation.cc:407] Connection header received
0808 11:47:02.752335 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.752339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.752413 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.752512 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.754762 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.756072 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.758483 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.759213 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.759342 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.762848 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.762853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.762856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.762886 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.766465 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.767355 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.767361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.767365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.767543 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.767948 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.767953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.767956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.768255 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.768530 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.768827 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.769072 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12562,"thread_start_us":12733,"threads_started":1}
I20260808 11:47:02.771320 1125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:47:02.781242 909 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.781611 909 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:02.781742 909 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a5759c9a4c4bd1815cf3f40406bdc1, State: Initialized, Role: FOLLOWER
I20260808 11:47:02.782400 909 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.783366 1162 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.783754 1162 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:47:02.784760 909 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:47:02.787678 1161 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:47:02.787830 1161 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:47:02.783290 1021 master_runner.cc:186] Error getting master registration for 127.13.158.126:38303: OK, instance_id { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" instance_seqno: 1786189621578423 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:47:02.795764 1021 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 11:47:02.805775 1185 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:47:02.806018 1185 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:47:02.811965 1012 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } attempt: 5
I20260808 11:47:02.803026 815 master_runner.cc:186] Error getting master registration for 127.13.158.125:44583: OK, instance_id { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" instance_seqno: 1786189622084916 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:47:02.814272 815 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 11:47:02.817950 842 tablet.cc:2369] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:02.822522 1145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
Aug 08 11:47:02 dist-test-slave-39l8 krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189622, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
I20260808 11:47:02.822839 1145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.824565 1145 leader_election.cc:288] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7a5759c9a4c4bd1815cf3f40406bdc1 (127.13.158.126:38303), d61fb27c1c074b29846a7cc18c7123f4 (127.13.158.125:44583)
I20260808 11:47:02.825666 873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97570e8105b84f969c245e1f8e7b11a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" is_pre_election: true
I20260808 11:47:02.826292 873 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97570e8105b84f969c245e1f8e7b11a8 in term 0.
I20260808 11:47:02.826928 1051 leader_election.cc:302] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97570e8105b84f969c245e1f8e7b11a8, e7a5759c9a4c4bd1815cf3f40406bdc1; no voters:
I20260808 11:47:02.827365 1145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:47:02.827527 1145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:47:02.827629 1145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:02.832810 1145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.833957 873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97570e8105b84f969c245e1f8e7b11a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1"
I20260808 11:47:02.834193 873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:02.837716 873 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97570e8105b84f969c245e1f8e7b11a8 in term 1.
I20260808 11:47:02.838240 1051 leader_election.cc:302] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97570e8105b84f969c245e1f8e7b11a8, e7a5759c9a4c4bd1815cf3f40406bdc1; no voters:
I20260808 11:47:02.839563 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.812641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:45943 (local address 127.0.0.1:38346)
0808 11:47:02.812988 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:02.813009 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:02.813121 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:02.814076 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:02.814081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:02.814620 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:02.814889 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.814902 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.816377 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.816384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.817899 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:02.817918 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.819900 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.819906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:02.820129 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.820837 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:02.820860 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:02.823586 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:02.835490 (+ 11904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.835513 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.835518 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.836098 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.836832 (+ 734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:02.836837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:02.836841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:02.836994 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:02.838182 (+ 1188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:02.838187 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:02.838441 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 11:47:02.838643 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 11:47:02.838987 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260808 11:47:02.841382 1145 leader_election.cc:288] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [CANDIDATE]: Term 1 election: Requested vote from peers e7a5759c9a4c4bd1815cf3f40406bdc1 (127.13.158.126:38303), d61fb27c1c074b29846a7cc18c7123f4 (127.13.158.125:44583)
I20260808 11:47:02.841596 1145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:47:02.842031 1145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [term 1 LEADER]: Becoming Leader. State: Replica: 97570e8105b84f969c245e1f8e7b11a8, State: Running, Role: LEADER
I20260808 11:47:02.842336 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:02.813357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.13.158.124:45943)
0808 11:47:02.813587 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:02.813593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:02.813617 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:47:02.813679 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:02.813684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:02.813760 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:02.813864 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:02.815064 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.816222 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.818049 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:02.818815 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:02.818952 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:02.823804 (+ 4852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:02.823811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:02.823820 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:02.823861 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:02.835276 (+ 11415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.836338 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.836344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.836348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.836626 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:02.837192 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:02.837197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:02.837200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:02.837931 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:02.840527 (+ 2596us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:02.840811 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 11:47:02.841292 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:47:02.842686 1145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.848901 1145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97570e8105b84f969c245e1f8e7b11a8. Latest consensus state: current_term: 1 leader_uuid: "97570e8105b84f969c245e1f8e7b11a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.849197 1145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:02.860085 1012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4: Bootstrap starting.
I20260808 11:47:02.865206 1012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:02.871315 1012 log.cc:824] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4: Log is configured to *not* fsync() on all Append() calls
W20260808 11:47:02.871980 1057 tablet.cc:2369] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:02.872614 1195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:47:02.874859 1195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:47:02.889029 1012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4: No bootstrap required, opened a new log
I20260808 11:47:02.895413 873 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [term 1 FOLLOWER]: Refusing update from remote peer 97570e8105b84f969c245e1f8e7b11a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:02.896449 1145 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:47:02.922665 1012 raft_consensus.cc:348] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.923988 1012 raft_consensus.cc:374] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:02.924358 1012 raft_consensus.cc:729] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61fb27c1c074b29846a7cc18c7123f4, State: Initialized, Role: FOLLOWER
I20260808 11:47:02.926173 1012 consensus_queue.cc:260] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } }
I20260808 11:47:02.929257 1162 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97570e8105b84f969c245e1f8e7b11a8. Latest consensus state: current_term: 1 leader_uuid: "97570e8105b84f969c245e1f8e7b11a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.929656 1162 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:47:02.932541 1202 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.932924 1202 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:47:02.934885 1012 sys_catalog.cc:564] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:47:02.934916 980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97570e8105b84f969c245e1f8e7b11a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61fb27c1c074b29846a7cc18c7123f4" is_pre_election: true
I20260808 11:47:02.935278 977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:02.935822 979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97570e8105b84f969c245e1f8e7b11a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61fb27c1c074b29846a7cc18c7123f4"
I20260808 11:47:02.939250 977 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [term 1 FOLLOWER]: Refusing update from remote peer 97570e8105b84f969c245e1f8e7b11a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:02.940171 1197 mvcc.cc:205] Tried to move back new op lower bound from 7316232695366623232 to 7316232695175933952. Current Snapshot: MvccSnapshot[applied={T|T < 7316232695366623232}]
I20260808 11:47:02.940986 1145 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:47:02.941744 1198 mvcc.cc:205] Tried to move back new op lower bound from 7316232695366623232 to 7316232695175933952. Current Snapshot: MvccSnapshot[applied={T|T < 7316232695366623232}]
I20260808 11:47:02.952069 1162 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97570e8105b84f969c245e1f8e7b11a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.952346 1162 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:47:02.954349 1192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97570e8105b84f969c245e1f8e7b11a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.954890 1192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:02.955921 1195 catalog_manager.cc:1347] Generated new cluster ID: 264350f70b834951b13473abd9480a65
I20260808 11:47:02.956048 1195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:47:02.981956 1202 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97570e8105b84f969c245e1f8e7b11a8. Latest consensus state: current_term: 1 leader_uuid: "97570e8105b84f969c245e1f8e7b11a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.982326 1202 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:47:02.982491 1212 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97570e8105b84f969c245e1f8e7b11a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.982967 1212 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:47:02.986093 1145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97570e8105b84f969c245e1f8e7b11a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7a5759c9a4c4bd1815cf3f40406bdc1" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38303 } } peers { permanent_uuid: "d61fb27c1c074b29846a7cc18c7123f4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 44583 } } peers { permanent_uuid: "97570e8105b84f969c245e1f8e7b11a8" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 45943 } } }
I20260808 11:47:02.986434 1145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:02.986665 1195 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:47:02.988955 1195 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:47:03.005285 1195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97570e8105b84f969c245e1f8e7b11a8: Generated new TSK 0
I20260808 11:47:03.006736 1195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:47:03.087690 1233 catalog_manager.cc:1259] Loaded cluster ID: 264350f70b834951b13473abd9480a65
I20260808 11:47:03.087815 1233 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4: loading cluster ID for follower catalog manager: success
I20260808 11:47:03.091539 1233 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4: acquiring CA information for follower catalog manager: success
I20260808 11:47:03.093962 1233 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d61fb27c1c074b29846a7cc18c7123f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:47:03.808145 1185 catalog_manager.cc:1259] Loaded cluster ID: 264350f70b834951b13473abd9480a65
I20260808 11:47:03.808265 1185 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: loading cluster ID for follower catalog manager: success
I20260808 11:47:03.810786 1185 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: acquiring CA information for follower catalog manager: success
I20260808 11:47:03.812517 1185 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e7a5759c9a4c4bd1815cf3f40406bdc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:47:04.132514 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 815
I20260808 11:47:04.133901 903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:04.242758 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 815
I20260808 11:47:04.274214 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 924
I20260808 11:47:04.275426 1004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:04.382817 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 924
I20260808 11:47:04.407507 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 1021
I20260808 11:47:04.408560 1116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:04.518044 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 1021
2026-08-08T11:47:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4993 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:47:04 dist-test-slave-39l8 krb5kdc[1247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:47:04 dist-test-slave-39l8 krb5kdc[1247](info): set up 2 sockets
Aug 08 11:47:04 dist-test-slave-39l8 krb5kdc[1247](info): commencing operation
krb5kdc: starting...
W20260808 11:47:05.810675 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:47:05 dist-test-slave-39l8 krb5kdc[1247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:47:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:47:06.090970 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:38201
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:38201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:06.422974 1266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:06.423333 1266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:06.423487 1266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:06.430867 1266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:06.431007 1266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:06.431110 1266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:06.431218 1266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:06.431303 1266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:06.441560 1266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:38201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:38201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:06.443027 1266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:06.444351 1266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:06.456861 1275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:06.456837 1276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:06.456771 1279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:06.457765 1266 server_base.cc:1034] running on GCE node
I20260808 11:47:06.458549 1266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:06.459993 1266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:06.461206 1266 hybrid_clock.cc:648] HybridClock initialized: now 1786189626461175 us; error 37 us; skew 500 ppm
I20260808 11:47:06.461565 1266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:06 dist-test-slave-39l8 krb5kdc[1247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189626, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:06.464761 1266 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:47:06.466357 1266 webserver.cc:460] Webserver started at http://127.13.158.126:36641/ using document root <none> and password file <none>
I20260808 11:47:06.467123 1266 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:06.467228 1266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:06.467496 1266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:06.470566 1266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "88f25298bf7c4a0e85a52efdd7584f67"
format_stamp: "Formatted at 2026-08-08 11:47:06 on dist-test-slave-39l8"
server_key: "4785d6513402f20ae433e2b8f2615603"
server_key_iv: "423fdad041cd5630c146f5c0bc01c5e9"
server_key_version: "encryptionkey@0"
I20260808 11:47:06.471359 1266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "88f25298bf7c4a0e85a52efdd7584f67"
format_stamp: "Formatted at 2026-08-08 11:47:06 on dist-test-slave-39l8"
server_key: "4785d6513402f20ae433e2b8f2615603"
server_key_iv: "423fdad041cd5630c146f5c0bc01c5e9"
server_key_version: "encryptionkey@0"
I20260808 11:47:06.476761 1266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:47:06.480212 1287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:06.481587 1266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:47:06.481776 1266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "88f25298bf7c4a0e85a52efdd7584f67"
format_stamp: "Formatted at 2026-08-08 11:47:06 on dist-test-slave-39l8"
server_key: "4785d6513402f20ae433e2b8f2615603"
server_key_iv: "423fdad041cd5630c146f5c0bc01c5e9"
server_key_version: "encryptionkey@0"
I20260808 11:47:06.481973 1266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:06.503901 1266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:06.507978 1266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:06.508262 1266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:06.527149 1266 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:38201
I20260808 11:47:06.527170 1364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:38201 every 8 connection(s)
I20260808 11:47:06.528592 1266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:47:06.532442 1365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:06.537405 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 1266
I20260808 11:47:06.537678 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:47:06.538522 13945 external_mini_cluster.cc:1428] Setting key 6daffc7b1e28d820ce19c892d84b7c29
I20260808 11:47:06.544174 1365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67: Bootstrap starting.
I20260808 11:47:06.548206 1365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:06.549521 1365 log.cc:824] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:06.552543 1365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67: No bootstrap required, opened a new log
Aug 08 11:47:06 dist-test-slave-39l8 krb5kdc[1247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:06.560559 1365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f25298bf7c4a0e85a52efdd7584f67" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38201 } }
I20260808 11:47:06.560945 1365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:06.561088 1365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88f25298bf7c4a0e85a52efdd7584f67, State: Initialized, Role: FOLLOWER
I20260808 11:47:06.561729 1365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f25298bf7c4a0e85a52efdd7584f67" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38201 } }
I20260808 11:47:06.561962 1365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:47:06.562165 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:06.540516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.13.158.126:38201)
0808 11:47:06.541403 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:06.541417 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:06.541461 (+ 44us) server_negotiation.cc:407] Connection header received
0808 11:47:06.543206 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:06.543231 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:06.543600 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:06.543921 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:06.545950 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:06.547363 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:06.549288 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:06.550036 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:06.550255 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:06.554567 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:06.554575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:06.554592 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:06.554627 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:06.557865 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:06.558610 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:06.558617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:06.558624 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:06.558882 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:06.559322 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:06.559327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:06.559330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:06.559831 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:06.560123 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:06.560576 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 11:47:06.561066 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":465,"threads_started":1}
I20260808 11:47:06.562136 1365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:47:06.562394 1365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:06.564911 1365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f25298bf7c4a0e85a52efdd7584f67" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38201 } }
I20260808 11:47:06.565471 1365 leader_election.cc:302] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88f25298bf7c4a0e85a52efdd7584f67; no voters:
I20260808 11:47:06.565917 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:47:06.566192 1365 leader_election.cc:288] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:47:06.566286 1375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:47:06.568676 1365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:47:06.571108 1375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [term 1 LEADER]: Becoming Leader. State: Replica: 88f25298bf7c4a0e85a52efdd7584f67, State: Running, Role: LEADER
I20260808 11:47:06.571767 1375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f25298bf7c4a0e85a52efdd7584f67" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38201 } }
I20260808 11:47:06.575013 1377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88f25298bf7c4a0e85a52efdd7584f67. Latest consensus state: current_term: 1 leader_uuid: "88f25298bf7c4a0e85a52efdd7584f67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f25298bf7c4a0e85a52efdd7584f67" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38201 } } }
I20260808 11:47:06.575101 1376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88f25298bf7c4a0e85a52efdd7584f67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f25298bf7c4a0e85a52efdd7584f67" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 38201 } } }
I20260808 11:47:06.575441 1377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:06.575532 1376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67 [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:06.577198 1387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:47:06.581714 1387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:47:06.590121 1387 catalog_manager.cc:1347] Generated new cluster ID: 3940dca2819348fabf7bbc3bd505ccbb
I20260808 11:47:06.590232 1387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:47:06.608611 1387 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:47:06.609916 1387 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:47:06.617714 1387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88f25298bf7c4a0e85a52efdd7584f67: Generated new TSK 0
I20260808 11:47:06.618589 1387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:47:06.627424 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 1266
I20260808 11:47:06.628588 1358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:06.718848 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 1266
2026-08-08T11:47:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2192 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:47:06 dist-test-slave-39l8 krb5kdc[1405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:47:06 dist-test-slave-39l8 krb5kdc[1405](info): set up 2 sockets
Aug 08 11:47:06 dist-test-slave-39l8 krb5kdc[1405](info): commencing operation
krb5kdc: starting...
W20260808 11:47:08.007645 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.005s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:47:08 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:47:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:47:08.292557 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36995
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:36995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:08.912063 1423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:08.912925 1423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:08.913255 1423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:08.929455 1423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:08.929714 1423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:08.929988 1423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:08.930269 1423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:08.930426 1423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:08.955862 1423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:36995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:08.958797 1423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:08.961330 1423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:08.983191 1428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:08.983292 1434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:08.983296 1429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:08.985384 1423 server_base.cc:1034] running on GCE node
I20260808 11:47:08.987134 1423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:08.989833 1423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:08.991405 1423 hybrid_clock.cc:648] HybridClock initialized: now 1786189628991320 us; error 67 us; skew 500 ppm
I20260808 11:47:08.992100 1423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:08 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189628, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:08.998075 1423 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:47:09.001132 1423 webserver.cc:460] Webserver started at http://127.13.158.126:35147/ using document root <none> and password file <none>
I20260808 11:47:09.002562 1423 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:09.002843 1423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:09.003358 1423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:09.009287 1423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "c453528a38e7477cb71a9906695d5e3a"
format_stamp: "Formatted at 2026-08-08 11:47:08 on dist-test-slave-39l8"
server_key: "96e5afa9fb577950669d2c90b335a90e"
server_key_iv: "7bca8ae3a0ff4e0ba110d3df1736a60a"
server_key_version: "encryptionkey@0"
I20260808 11:47:09.010939 1423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "c453528a38e7477cb71a9906695d5e3a"
format_stamp: "Formatted at 2026-08-08 11:47:08 on dist-test-slave-39l8"
server_key: "96e5afa9fb577950669d2c90b335a90e"
server_key_iv: "7bca8ae3a0ff4e0ba110d3df1736a60a"
server_key_version: "encryptionkey@0"
I20260808 11:47:09.021430 1423 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.008s sys 0.000s
I20260808 11:47:09.027402 1443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:09.029930 1423 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:47:09.030146 1423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c453528a38e7477cb71a9906695d5e3a"
format_stamp: "Formatted at 2026-08-08 11:47:08 on dist-test-slave-39l8"
server_key: "96e5afa9fb577950669d2c90b335a90e"
server_key_iv: "7bca8ae3a0ff4e0ba110d3df1736a60a"
server_key_version: "encryptionkey@0"
I20260808 11:47:09.030474 1423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:09.074687 1423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:09.082217 1423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:09.082685 1423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:09.118459 1423 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:36995
I20260808 11:47:09.118508 1523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:36995 every 8 connection(s)
I20260808 11:47:09.121096 1423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:47:09.122586 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 1423
I20260808 11:47:09.123159 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:47:09.124379 13945 external_mini_cluster.cc:1428] Setting key bccf8583d17d537a4cb706ba991f8324
I20260808 11:47:09.129647 1524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:47:09 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:09.145424 1524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Bootstrap starting.
I20260808 11:47:09.149704 1524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:09.151067 1524 log.cc:824] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:09.154520 1524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: No bootstrap required, opened a new log
I20260808 11:47:09.154778 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:09.128054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.13.158.126:36995)
0808 11:47:09.129662 (+ 1608us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:09.129703 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:09.129810 (+ 107us) server_negotiation.cc:407] Connection header received
0808 11:47:09.134744 (+ 4934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:09.134801 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:09.135500 (+ 699us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:09.135946 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:09.137346 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:09.139478 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:09.140853 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:09.141879 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:09.142079 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:09.146204 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:09.146216 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:09.146235 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:09.146280 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:09.150295 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:09.151061 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:09.151071 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:09.151080 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:09.151294 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:09.151730 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:09.151737 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:09.151741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:09.152302 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:09.152645 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:09.153026 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 11:47:09.153470 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1083,"thread_start_us":574,"threads_started":1}
I20260808 11:47:09.163082 1524 raft_consensus.cc:348] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:09.163431 1524 raft_consensus.cc:374] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:09.163503 1524 raft_consensus.cc:729] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c453528a38e7477cb71a9906695d5e3a, State: Initialized, Role: FOLLOWER
I20260808 11:47:09.163977 1524 consensus_queue.cc:260] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:09.164167 1524 raft_consensus.cc:388] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:47:09.164269 1524 raft_consensus.cc:482] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:47:09.164395 1524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:09.166473 1524 raft_consensus.cc:504] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:09.167160 1524 leader_election.cc:302] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c453528a38e7477cb71a9906695d5e3a; no voters:
I20260808 11:47:09.167703 1524 leader_election.cc:288] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:47:09.167863 1532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:47:09.168587 1532 raft_consensus.cc:686] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 1 LEADER]: Becoming Leader. State: Replica: c453528a38e7477cb71a9906695d5e3a, State: Running, Role: LEADER
I20260808 11:47:09.169117 1532 consensus_queue.cc:237] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:09.169878 1524 sys_catalog.cc:564] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:47:09.171984 1533 sys_catalog.cc:455] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c453528a38e7477cb71a9906695d5e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } } }
I20260808 11:47:09.172360 1534 sys_catalog.cc:455] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c453528a38e7477cb71a9906695d5e3a. Latest consensus state: current_term: 1 leader_uuid: "c453528a38e7477cb71a9906695d5e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } } }
I20260808 11:47:09.174435 1533 sys_catalog.cc:458] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:09.174458 1534 sys_catalog.cc:458] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:09.175308 1549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:47:09.181632 1549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:47:09.192754 1549 catalog_manager.cc:1347] Generated new cluster ID: 446933dd9175472f802578c280d2b8d1
I20260808 11:47:09.192901 1549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:47:09.205981 1549 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:47:09.207796 1549 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:47:09.219568 1549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Generated new TSK 0
I20260808 11:47:09.220409 1549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:47:09.292382 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36995
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:09.633368 1560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:09.633651 1560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:09.633741 1560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:09.641038 1560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:09.641135 1560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:09.641274 1560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:47:09.650826 1560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:09.652081 1560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:09.653475 1560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:09.668146 1572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:09.668114 1568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:09.668300 1567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:09.669170 1560 server_base.cc:1034] running on GCE node
I20260808 11:47:09.669744 1560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:09.670749 1560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:09.672012 1560 hybrid_clock.cc:648] HybridClock initialized: now 1786189629671948 us; error 47 us; skew 500 ppm
I20260808 11:47:09.672402 1560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:09 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189629, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:09.678658 1560 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:47:09.680178 1560 webserver.cc:460] Webserver started at http://127.13.158.65:34567/ using document root <none> and password file <none>
I20260808 11:47:09.680891 1560 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:09.681042 1560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:09.681352 1560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:09.684204 1560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc"
format_stamp: "Formatted at 2026-08-08 11:47:09 on dist-test-slave-39l8"
server_key: "86e0053708335184a64ee4631c168174"
server_key_iv: "f58e9389d820414e1f24ec6791319563"
server_key_version: "encryptionkey@0"
I20260808 11:47:09.685050 1560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc"
format_stamp: "Formatted at 2026-08-08 11:47:09 on dist-test-slave-39l8"
server_key: "86e0053708335184a64ee4631c168174"
server_key_iv: "f58e9389d820414e1f24ec6791319563"
server_key_version: "encryptionkey@0"
I20260808 11:47:09.690004 1560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:47:09.693472 1583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:09.694669 1560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:47:09.694900 1560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc"
format_stamp: "Formatted at 2026-08-08 11:47:09 on dist-test-slave-39l8"
server_key: "86e0053708335184a64ee4631c168174"
server_key_iv: "f58e9389d820414e1f24ec6791319563"
server_key_version: "encryptionkey@0"
I20260808 11:47:09.695103 1560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:09.722110 1560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:09.725869 1560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:09.726166 1560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:09.727175 1560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:09.728480 1560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:09.728577 1560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:09.728736 1560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:09.728829 1560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:09.761911 1560 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:34571
I20260808 11:47:09.761955 1736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:34571 every 8 connection(s)
I20260808 11:47:09.763255 1560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:47:09.770015 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 1560
I20260808 11:47:09.770337 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:47:09.771114 13945 external_mini_cluster.cc:1428] Setting key acca2f1d22197bae8c64ce49363cab5e
Aug 08 11:47:09 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189629, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:09.783686 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:09.765713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:32857 (local address 127.13.158.126:36995)
0808 11:47:09.766203 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:09.766209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:09.767222 (+ 1013us) server_negotiation.cc:407] Connection header received
0808 11:47:09.769039 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:09.769044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:09.769131 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:09.769273 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:09.771027 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:09.772063 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:09.773701 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:09.774234 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:09.774375 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:09.777897 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:09.777904 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:09.777908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:09.777949 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:09.780524 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:09.781249 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:09.781255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:09.781258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:09.781393 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:09.781835 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:09.781840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:09.781849 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:09.782097 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:09.782274 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:09.783011 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 11:47:09.783232 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":252,"threads_started":1}
I20260808 11:47:09.784643 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:09.766296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:32857)
0808 11:47:09.767037 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:09.767086 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:09.768761 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:09.769515 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:09.769523 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:09.770016 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:09.770855 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:09.770876 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:09.772245 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:09.772249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:09.773535 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:09.773548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:09.774475 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:09.774483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:09.774769 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:09.775465 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:09.775490 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:09.777679 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:09.780677 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:09.780686 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:09.780704 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:09.781116 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:09.781520 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:09.781525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:09.781528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:09.781716 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:09.782254 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:09.782263 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:09.782603 (+ 340us) client_negotiation.cc:769] Sending connection context
0808 11:47:09.782906 (+ 303us) client_negotiation.cc:240] Negotiation successful
0808 11:47:09.783453 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":275,"threads_started":1}
I20260808 11:47:09.786245 1737 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36995
I20260808 11:47:09.786576 1737 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:09.787511 1737 heartbeater.cc:511] Master 127.13.158.126:36995 requested a full tablet report, sending...
I20260808 11:47:09.790004 1470 ts_manager.cc:194] Registered new tserver with Master: 4d6c52a31c6e498f81c9b9ec80ad67cc (127.13.158.65:34571)
I20260808 11:47:09.792268 1470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:32857
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:47:09.836190 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36995
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:10.184949 1744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:10.185252 1744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:10.185410 1744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:10.192674 1744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:10.192857 1744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:10.193029 1744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:47:10.202971 1744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:10.204272 1744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:10.205587 1744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:10.219084 1757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:10.219180 1752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:10.219177 1753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:10.220044 1744 server_base.cc:1034] running on GCE node
I20260808 11:47:10.220674 1744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:10.221617 1744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:10.222828 1744 hybrid_clock.cc:648] HybridClock initialized: now 1786189630222786 us; error 52 us; skew 500 ppm
I20260808 11:47:10.223155 1744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:10 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:10.226413 1744 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:47:10.227947 1744 webserver.cc:460] Webserver started at http://127.13.158.66:45421/ using document root <none> and password file <none>
I20260808 11:47:10.228673 1744 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:10.228811 1744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:10.229135 1744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:10.232205 1744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "3130deefc3234804bf4185e3d47c1104"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "16225f1652b1b25db94182cc74d70325"
server_key_iv: "71753af7df8d9c23195037eaebf565b3"
server_key_version: "encryptionkey@0"
I20260808 11:47:10.233083 1744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "3130deefc3234804bf4185e3d47c1104"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "16225f1652b1b25db94182cc74d70325"
server_key_iv: "71753af7df8d9c23195037eaebf565b3"
server_key_version: "encryptionkey@0"
I20260808 11:47:10.238317 1744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:47:10.241822 1768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:10.243522 1744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:47:10.243726 1744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "3130deefc3234804bf4185e3d47c1104"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "16225f1652b1b25db94182cc74d70325"
server_key_iv: "71753af7df8d9c23195037eaebf565b3"
server_key_version: "encryptionkey@0"
I20260808 11:47:10.243955 1744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:10.277127 1744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:10.281783 1744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:10.282109 1744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:10.283248 1744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:10.284710 1744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:10.284782 1744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:10.284952 1744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:10.285002 1744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:10.318908 1744 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:44801
I20260808 11:47:10.318953 1929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:44801 every 8 connection(s)
I20260808 11:47:10.320312 1744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:47:10.324241 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 1744
I20260808 11:47:10.324426 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:47:10.324859 13945 external_mini_cluster.cc:1428] Setting key 3c08753c789b9877936ba8e65efd290f
Aug 08 11:47:10 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:10.342041 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.322883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:36285 (local address 127.13.158.126:36995)
0808 11:47:10.323321 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:10.323325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:10.324214 (+ 889us) server_negotiation.cc:407] Connection header received
0808 11:47:10.326321 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:10.326326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:10.326418 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:10.326534 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:10.328046 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.328775 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.330402 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.330894 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.330971 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.335286 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:10.335291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:10.335295 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:10.335328 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:10.338442 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:10.339430 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:10.339438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:10.339444 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:10.339607 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:10.340032 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:10.340040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:10.340046 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:10.340409 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:10.340568 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:10.341254 (+ 686us) server_negotiation.cc:299] Negotiation successful
0808 11:47:10.341533 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":215,"threads_started":1}
I20260808 11:47:10.342782 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.323387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:36285)
0808 11:47:10.324036 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:10.324082 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:10.326028 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:10.326817 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:10.326825 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:10.327326 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:10.327877 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.327893 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.328929 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.328941 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.330198 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.330218 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.331256 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.331261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.331460 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.332195 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:10.332223 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:10.335034 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:10.338599 (+ 3565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:10.338605 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:10.338617 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:10.339296 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:10.339719 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:10.339723 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:10.339725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:10.339907 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:10.340543 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:10.340548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:10.340866 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 11:47:10.341100 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 11:47:10.341560 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":265,"threads_started":1}
I20260808 11:47:10.344404 1930 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36995
I20260808 11:47:10.344728 1930 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:10.345477 1930 heartbeater.cc:511] Master 127.13.158.126:36995 requested a full tablet report, sending...
I20260808 11:47:10.347311 1470 ts_manager.cc:194] Registered new tserver with Master: 3130deefc3234804bf4185e3d47c1104 (127.13.158.66:44801)
I20260808 11:47:10.348603 1470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:36285
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:47:10.390003 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36995
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:10.726249 1939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:10.726531 1939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:10.726758 1939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:10.733958 1939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:10.734059 1939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:10.734252 1939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:47:10.744451 1939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:10.745690 1939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:10.747067 1939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:10.760212 1949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:10.760182 1946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:10.760500 1945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:10.760968 1939 server_base.cc:1034] running on GCE node
I20260808 11:47:10.761574 1939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:10.762576 1939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:10.763813 1939 hybrid_clock.cc:648] HybridClock initialized: now 1786189630763757 us; error 50 us; skew 500 ppm
I20260808 11:47:10.764124 1939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:10 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:10.767393 1939 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:47:10.768759 1939 webserver.cc:460] Webserver started at http://127.13.158.67:44937/ using document root <none> and password file <none>
I20260808 11:47:10.769452 1939 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:10.769613 1939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:10.769974 1939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:10.772972 1939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "6c8c11d305b540309f1baed8738c9c94"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "269ba3c5f36aa1e47a398daaa66647b2"
server_key_iv: "a35affc198cea11ddbc016931a129fb5"
server_key_version: "encryptionkey@0"
I20260808 11:47:10.773841 1939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "6c8c11d305b540309f1baed8738c9c94"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "269ba3c5f36aa1e47a398daaa66647b2"
server_key_iv: "a35affc198cea11ddbc016931a129fb5"
server_key_version: "encryptionkey@0"
I20260808 11:47:10.778980 1939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 11:47:10.782397 1956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:10.783809 1939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:10.783994 1939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "6c8c11d305b540309f1baed8738c9c94"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "269ba3c5f36aa1e47a398daaa66647b2"
server_key_iv: "a35affc198cea11ddbc016931a129fb5"
server_key_version: "encryptionkey@0"
I20260808 11:47:10.784215 1939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:10.796144 1737 heartbeater.cc:503] Master 127.13.158.126:36995 was elected leader, sending a full tablet report...
I20260808 11:47:10.798640 1939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:10.803023 1939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:10.803261 1939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:10.804167 1939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:10.805410 1939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:10.805487 1939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:10.805575 1939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:10.805620 1939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:10.840059 1939 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:33489
I20260808 11:47:10.840109 2129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:33489 every 8 connection(s)
I20260808 11:47:10.841390 1939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:47:10.847889 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 1939
I20260808 11:47:10.848086 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:47:10.848701 13945 external_mini_cluster.cc:1428] Setting key 0cb189efd9408bce5013a7808c4c6d98
Aug 08 11:47:10 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:10.861280 2133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.844003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:48103 (local address 127.13.158.126:36995)
0808 11:47:10.844427 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:10.844432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:10.845421 (+ 989us) server_negotiation.cc:407] Connection header received
0808 11:47:10.847398 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:10.847402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:10.847494 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:10.847613 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:10.849670 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.850464 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.852112 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.852540 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.852676 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.856038 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:10.856042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:10.856045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:10.856077 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:10.858776 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:10.859428 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:10.859433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:10.859435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:10.859572 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:10.859938 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:10.859941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:10.859943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:10.860191 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:10.860303 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:10.860879 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 11:47:10.861027 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":199,"threads_started":1}
I20260808 11:47:10.862319 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.844541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:48103)
0808 11:47:10.845268 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:10.845305 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:10.847096 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:10.847925 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:10.847937 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:10.848669 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:10.849474 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.849501 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.850613 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.850619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.851935 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.851948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.852792 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.852800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.853030 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.853740 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:10.853763 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:10.855854 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:10.858902 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:10.858908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:10.858919 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:10.859314 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:10.859661 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:10.859664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:10.859666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:10.859841 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:10.860291 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:10.860296 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:10.860586 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 11:47:10.860809 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 11:47:10.861178 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":261,"threads_started":1}
I20260808 11:47:10.863916 2130 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36995
I20260808 11:47:10.864236 2130 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:10.865115 2130 heartbeater.cc:511] Master 127.13.158.126:36995 requested a full tablet report, sending...
I20260808 11:47:10.866521 1470 ts_manager.cc:194] Registered new tserver with Master: 6c8c11d305b540309f1baed8738c9c94 (127.13.158.67:33489)
I20260808 11:47:10.867355 1470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:48103
I20260808 11:47:10.878156 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:47:10.895569 2133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.882878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46572 (local address 127.13.158.126:36995)
0808 11:47:10.883041 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:10.883046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:10.883584 (+ 538us) server_negotiation.cc:407] Connection header received
0808 11:47:10.883731 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:10.883735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:10.883817 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:10.883898 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:10.885238 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.885933 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.887301 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.887719 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.887808 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.890295 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:10.890302 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:10.890318 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:10.890355 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:10.892794 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:10.893528 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:10.893534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:10.893537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:10.893669 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:10.894090 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:10.894095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:10.894098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:10.894417 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:10.894666 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:10.895010 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 11:47:10.895271 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:47:10.899721 1470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:47:10.906652 1470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:47:10.930603 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.919883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.13.158.65:34571)
0808 11:47:10.923407 (+ 3524us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:10.923413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:10.923434 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:47:10.923541 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:10.923548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:10.923766 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:10.923971 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:10.924706 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.926109 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.928273 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.929473 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.929618 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.929889 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:10.930124 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 11:47:10.930283 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3388,"thread_start_us":3239,"threads_started":1}
I20260808 11:47:10.930727 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.919392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:34571 (local address 127.0.0.1:51022)
0808 11:47:10.920218 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:10.920276 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:10.920455 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:10.923988 (+ 3533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:10.923992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:10.924015 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:10.924497 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.924510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.926269 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.926274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.928053 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.928067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.929675 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.929681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.929881 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.929932 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 11:47:10.930071 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 11:47:10.930268 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":46,"thread_start_us":1367,"threads_started":1}
I20260808 11:47:10.931484 2161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.920672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:33489 (local address 127.0.0.1:48616)
0808 11:47:10.921636 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:10.921658 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:10.921763 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:10.923123 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:10.923130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:10.923167 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:10.923628 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.923647 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.925974 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.925978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.928612 (+ 2634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.928625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.930517 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.930521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.930834 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.930874 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 11:47:10.930956 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 11:47:10.931086 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":156,"thread_start_us":758,"threads_started":1}
I20260808 11:47:10.932066 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.919512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:44801 (local address 127.0.0.1:46610)
0808 11:47:10.921019 (+ 1507us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:10.921042 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:10.921159 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:10.923547 (+ 2388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:10.923552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:10.923579 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:10.923980 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.923993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.925774 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.925781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.928609 (+ 2828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:10.928625 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.930504 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.930511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:10.931079 (+ 568us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.931108 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 11:47:10.931195 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 11:47:10.931315 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1388,"mutex_wait_us":435,"thread_start_us":327,"threads_started":1}
I20260808 11:47:10.932346 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.919980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.13.158.66:44801)
0808 11:47:10.922979 (+ 2999us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:10.922989 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:10.923017 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:47:10.923112 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:10.923118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:10.923344 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:10.923546 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:10.924218 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.925639 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.928802 (+ 3163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.930246 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.931563 (+ 1317us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.931753 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:10.931825 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 11:47:10.931989 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2842,"thread_start_us":2744,"threads_started":1}
I20260808 11:47:10.934434 1649 tablet_service.cc:1467] Processing CreateTablet for tablet a12c524d39bd4f498c8ccb6bd67e9176 (DEFAULT_TABLE table=test-table [id=a5d02abec6984c0fb803adbe68b99672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:10.935938 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:10.921253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.13.158.67:33489)
0808 11:47:10.922450 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:10.922459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:10.922491 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:47:10.922585 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:10.922593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:10.922894 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:10.923850 (+ 956us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:10.924152 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.925836 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.928802 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:10.930227 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:10.935141 (+ 4914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:10.935391 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:10.935460 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 11:47:10.935611 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1012,"thread_start_us":1132,"threads_started":1}
I20260808 11:47:10.935988 1649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12c524d39bd4f498c8ccb6bd67e9176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:10.938628 1829 tablet_service.cc:1467] Processing CreateTablet for tablet a12c524d39bd4f498c8ccb6bd67e9176 (DEFAULT_TABLE table=test-table [id=a5d02abec6984c0fb803adbe68b99672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:10.938815 2033 tablet_service.cc:1467] Processing CreateTablet for tablet a12c524d39bd4f498c8ccb6bd67e9176 (DEFAULT_TABLE table=test-table [id=a5d02abec6984c0fb803adbe68b99672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:10.940029 1829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12c524d39bd4f498c8ccb6bd67e9176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:10.940120 2033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12c524d39bd4f498c8ccb6bd67e9176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:10.950933 2163 tablet_bootstrap.cc:492] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Bootstrap starting.
I20260808 11:47:10.952749 2165 tablet_bootstrap.cc:492] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Bootstrap starting.
I20260808 11:47:10.954536 2167 tablet_bootstrap.cc:492] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Bootstrap starting.
I20260808 11:47:10.955825 2163 tablet_bootstrap.cc:654] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:10.956676 2165 tablet_bootstrap.cc:654] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:10.957144 2163 log.cc:824] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:10.957703 2165 log.cc:824] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:10.958053 2167 tablet_bootstrap.cc:654] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:10.959317 2167 log.cc:824] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:10.960354 2163 tablet_bootstrap.cc:492] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: No bootstrap required, opened a new log
I20260808 11:47:10.960631 2163 ts_tablet_manager.cc:1397] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260808 11:47:10.960813 2165 tablet_bootstrap.cc:492] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: No bootstrap required, opened a new log
I20260808 11:47:10.961125 2165 ts_tablet_manager.cc:1397] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 11:47:10.962630 2167 tablet_bootstrap.cc:492] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: No bootstrap required, opened a new log
I20260808 11:47:10.962893 2167 ts_tablet_manager.cc:1397] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 11:47:10.969484 2167 raft_consensus.cc:348] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:10.969859 2167 raft_consensus.cc:374] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:10.970031 2167 raft_consensus.cc:729] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3130deefc3234804bf4185e3d47c1104, State: Initialized, Role: FOLLOWER
I20260808 11:47:10.970652 2167 consensus_queue.cc:260] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:10.971547 2163 raft_consensus.cc:348] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:10.971607 2165 raft_consensus.cc:348] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:10.971930 1930 heartbeater.cc:503] Master 127.13.158.126:36995 was elected leader, sending a full tablet report...
I20260808 11:47:10.971987 2165 raft_consensus.cc:374] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:10.972065 2163 raft_consensus.cc:374] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:10.972116 2165 raft_consensus.cc:729] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8c11d305b540309f1baed8738c9c94, State: Initialized, Role: FOLLOWER
I20260808 11:47:10.972155 2163 raft_consensus.cc:729] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6c52a31c6e498f81c9b9ec80ad67cc, State: Initialized, Role: FOLLOWER
I20260808 11:47:10.972643 2167 ts_tablet_manager.cc:1428] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 11:47:10.972702 2165 consensus_queue.cc:260] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:10.972872 2163 consensus_queue.cc:260] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:10.975486 2163 ts_tablet_manager.cc:1428] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260808 11:47:10.976400 2130 heartbeater.cc:503] Master 127.13.158.126:36995 was elected leader, sending a full tablet report...
I20260808 11:47:10.977018 2165 ts_tablet_manager.cc:1428] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Time spent starting tablet: real 0.016s user 0.006s sys 0.009s
W20260808 11:47:11.015650 1738 tablet.cc:2369] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:47:11.072365 1933 tablet.cc:2369] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:11.086541 2176 raft_consensus.cc:482] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:47:11.086894 2176 raft_consensus.cc:504] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:11.088644 2176 leader_election.cc:288] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3130deefc3234804bf4185e3d47c1104 (127.13.158.66:44801), 6c8c11d305b540309f1baed8738c9c94 (127.13.158.67:33489)
W20260808 11:47:11.093133 2132 tablet.cc:2369] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:11.096248 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:11.090132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:52875 (local address 127.13.158.67:33489)
0808 11:47:11.090563 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:11.090571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:11.090652 (+ 81us) server_negotiation.cc:407] Connection header received
0808 11:47:11.090859 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:11.090865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:11.090943 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:11.091032 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:11.091617 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.092505 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.094382 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.095257 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.095388 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:11.095467 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:11.095821 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 11:47:11.095966 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260808 11:47:11.096345 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:11.089983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:33489 (local address 127.13.158.65:52875)
0808 11:47:11.090563 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:11.090576 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:11.090674 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:11.091062 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:11.091066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:11.091088 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:11.091462 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:11.091474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.092644 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.092648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:11.094202 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:11.094215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.095479 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.095485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:11.095660 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:11.095684 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:47:11.095763 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 11:47:11.095885 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":25,"thread_start_us":249,"threads_started":1}
I20260808 11:47:11.096402 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:11.089136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:44801 (local address 127.13.158.65:33673)
0808 11:47:11.089815 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:11.089830 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:11.090011 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:11.090386 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:11.090393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:11.090431 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:11.090907 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:11.090921 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.092164 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.092169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:11.094278 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:11.094316 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.095588 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.095592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:11.095714 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:11.095736 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:47:11.095848 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 11:47:11.095979 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":27,"thread_start_us":366,"threads_started":1}
I20260808 11:47:11.096298 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:11.089163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:33673 (local address 127.13.158.66:44801)
0808 11:47:11.089426 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:11.089435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:11.090040 (+ 605us) server_negotiation.cc:407] Connection header received
0808 11:47:11.090134 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:11.090141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:11.090240 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:11.090341 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:11.091081 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.092036 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.094464 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.095363 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.095530 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:11.095602 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:11.095892 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 11:47:11.096048 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:47:11.097580 2063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12c524d39bd4f498c8ccb6bd67e9176" candidate_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8c11d305b540309f1baed8738c9c94" is_pre_election: true
I20260808 11:47:11.097580 1862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12c524d39bd4f498c8ccb6bd67e9176" candidate_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3130deefc3234804bf4185e3d47c1104" is_pre_election: true
I20260808 11:47:11.098188 2063 raft_consensus.cc:2454] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6c52a31c6e498f81c9b9ec80ad67cc in term 0.
I20260808 11:47:11.098177 1862 raft_consensus.cc:2454] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6c52a31c6e498f81c9b9ec80ad67cc in term 0.
I20260808 11:47:11.099088 1585 leader_election.cc:302] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6c52a31c6e498f81c9b9ec80ad67cc, 6c8c11d305b540309f1baed8738c9c94; no voters:
I20260808 11:47:11.099537 2176 raft_consensus.cc:2781] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:47:11.099699 2176 raft_consensus.cc:482] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:47:11.099818 2176 raft_consensus.cc:3037] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:11.101951 2176 raft_consensus.cc:504] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:11.102545 2176 leader_election.cc:288] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [CANDIDATE]: Term 1 election: Requested vote from peers 3130deefc3234804bf4185e3d47c1104 (127.13.158.66:44801), 6c8c11d305b540309f1baed8738c9c94 (127.13.158.67:33489)
I20260808 11:47:11.103116 1862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12c524d39bd4f498c8ccb6bd67e9176" candidate_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3130deefc3234804bf4185e3d47c1104"
I20260808 11:47:11.103169 2063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12c524d39bd4f498c8ccb6bd67e9176" candidate_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8c11d305b540309f1baed8738c9c94"
I20260808 11:47:11.103317 1862 raft_consensus.cc:3037] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:11.103366 2063 raft_consensus.cc:3037] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:11.105500 1862 raft_consensus.cc:2454] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6c52a31c6e498f81c9b9ec80ad67cc in term 1.
I20260808 11:47:11.105510 2063 raft_consensus.cc:2454] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6c52a31c6e498f81c9b9ec80ad67cc in term 1.
I20260808 11:47:11.105906 1585 leader_election.cc:302] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3130deefc3234804bf4185e3d47c1104, 4d6c52a31c6e498f81c9b9ec80ad67cc; no voters:
I20260808 11:47:11.106189 2176 raft_consensus.cc:2781] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:47:11.106653 2176 raft_consensus.cc:686] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 LEADER]: Becoming Leader. State: Replica: 4d6c52a31c6e498f81c9b9ec80ad67cc, State: Running, Role: LEADER
I20260808 11:47:11.107177 2176 consensus_queue.cc:237] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } }
I20260808 11:47:11.111435 1468 catalog_manager.cc:5495] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d6c52a31c6e498f81c9b9ec80ad67cc (127.13.158.65). New cstate: current_term: 1 leader_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } health_report { overall_health: UNKNOWN } } }
I20260808 11:47:11.171965 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:11.166509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.13.158.65:34571)
0808 11:47:11.166737 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:11.166742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:11.166800 (+ 58us) server_negotiation.cc:407] Connection header received
0808 11:47:11.166892 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:11.166896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:11.166981 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:11.167103 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:47:11.167534 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.168261 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.169828 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.170196 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.170282 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:11.170908 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:47:11.171397 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:47:11.171505 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:11.171607 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 11:47:11.171676 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:47:11.180315 2063 raft_consensus.cc:1261] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 1 FOLLOWER]: Refusing update from remote peer 4d6c52a31c6e498f81c9b9ec80ad67cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:11.180461 1862 raft_consensus.cc:1261] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 1 FOLLOWER]: Refusing update from remote peer 4d6c52a31c6e498f81c9b9ec80ad67cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:11.181352 2182 consensus_queue.cc:1035] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:47:11.181716 2176 consensus_queue.cc:1035] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:47:11.192456 2185 mvcc.cc:205] Tried to move back new op lower bound from 7316232729304813568 to 7316232729018601472. Current Snapshot: MvccSnapshot[applied={T|T < 7316232729304813568}]
I20260808 11:47:11.208032 1468 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:47:11.208314 1468 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:47:11.211973 1468 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Sending DeleteTablet for 3 replicas of tablet a12c524d39bd4f498c8ccb6bd67e9176
I20260808 11:47:11.213130 1649 tablet_service.cc:1514] Processing DeleteTablet for tablet a12c524d39bd4f498c8ccb6bd67e9176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:11 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:51022
I20260808 11:47:11.213258 1829 tablet_service.cc:1514] Processing DeleteTablet for tablet a12c524d39bd4f498c8ccb6bd67e9176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:11 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:46610
I20260808 11:47:11.213415 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 1560
I20260808 11:47:11.214129 2033 tablet_service.cc:1514] Processing DeleteTablet for tablet a12c524d39bd4f498c8ccb6bd67e9176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:11 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:48616
I20260808 11:47:11.233906 2203 tablet_replica.cc:330] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: stopping tablet replica
I20260808 11:47:11.234127 2205 tablet_replica.cc:330] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: stopping tablet replica
I20260808 11:47:11.234447 2203 raft_consensus.cc:2229] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:47:11.234488 2205 raft_consensus.cc:2229] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:47:11.235023 2205 raft_consensus.cc:2258] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:11.235023 2203 raft_consensus.cc:2258] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:11.235822 2202 tablet_replica.cc:330] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: stopping tablet replica
I20260808 11:47:11.236243 2202 raft_consensus.cc:2229] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:47:11.236981 2202 raft_consensus.cc:2258] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:11.239468 2205 ts_tablet_manager.cc:1905] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:11.239521 2203 ts_tablet_manager.cc:1905] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 11:47:11 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:47:11.243187 2202 ts_tablet_manager.cc:1905] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:11.246835 2205 ts_tablet_manager.cc:1918] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:11.246892 2203 ts_tablet_manager.cc:1918] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:11.247040 2205 log.cc:1196] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/a12c524d39bd4f498c8ccb6bd67e9176
I20260808 11:47:11.247112 2203 log.cc:1196] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/a12c524d39bd4f498c8ccb6bd67e9176
I20260808 11:47:11.247740 2203 ts_tablet_manager.cc:1939] T a12c524d39bd4f498c8ccb6bd67e9176 P 3130deefc3234804bf4185e3d47c1104: Deleting consensus metadata
I20260808 11:47:11.247750 2205 ts_tablet_manager.cc:1939] T a12c524d39bd4f498c8ccb6bd67e9176 P 6c8c11d305b540309f1baed8738c9c94: Deleting consensus metadata
I20260808 11:47:11.249747 1449 catalog_manager.cc:4841] TS 3130deefc3234804bf4185e3d47c1104 (127.13.158.66:44801): tablet a12c524d39bd4f498c8ccb6bd67e9176 (table test-table [id=a5d02abec6984c0fb803adbe68b99672]) successfully deleted
I20260808 11:47:11.250348 1449 catalog_manager.cc:4841] TS 6c8c11d305b540309f1baed8738c9c94 (127.13.158.67:33489): tablet a12c524d39bd4f498c8ccb6bd67e9176 (table test-table [id=a5d02abec6984c0fb803adbe68b99672]) successfully deleted
I20260808 11:47:11.252774 2202 ts_tablet_manager.cc:1918] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:11.252976 2202 log.cc:1196] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/a12c524d39bd4f498c8ccb6bd67e9176
I20260808 11:47:11.253612 2202 ts_tablet_manager.cc:1939] T a12c524d39bd4f498c8ccb6bd67e9176 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Deleting consensus metadata
I20260808 11:47:11.255368 1445 catalog_manager.cc:4841] TS 4d6c52a31c6e498f81c9b9ec80ad67cc (127.13.158.65:34571): tablet a12c524d39bd4f498c8ccb6bd67e9176 (table test-table [id=a5d02abec6984c0fb803adbe68b99672]) successfully deleted
I20260808 11:47:11.255966 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:11.215008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.13.158.65:34571)
0808 11:47:11.215716 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:11.215726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:11.225416 (+ 9690us) server_negotiation.cc:407] Connection header received
0808 11:47:11.225968 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:11.225978 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:11.226096 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:11.226206 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:11.228817 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.230452 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.235449 (+ 4997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.236197 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.236339 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:11.246764 (+ 10425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:11.246780 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:11.246799 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:11.246866 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:11.251667 (+ 4801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:11.252645 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:11.252654 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:11.252658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:11.252841 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:11.253357 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:11.253363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:11.253368 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:11.253867 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:11.254210 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:11.254845 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 11:47:11.255573 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542}
I20260808 11:47:11.256624 1731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:11.367897 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 1560
I20260808 11:47:11.395040 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 1744
Aug 08 11:47:11 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:47:11.410981 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:11.395888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.13.158.66:44801)
0808 11:47:11.396135 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:11.396141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:11.396161 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:47:11.396355 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:11.396360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:11.396429 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:11.396539 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:11.397877 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.398653 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.400177 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.400601 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.400749 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:11.405282 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:11.405300 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:11.405317 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:11.405364 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:11.408277 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:11.409010 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:11.409017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:11.409020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:11.409148 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:11.409488 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:11.409491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:11.409493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:11.409896 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:11.410110 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:11.410392 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 11:47:11.410560 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260808 11:47:11.411677 1924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:11.500350 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 1744
I20260808 11:47:11.526405 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 1939
Aug 08 11:47:11 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:47:11.541524 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:11.527295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.13.158.67:33489)
0808 11:47:11.527470 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:11.527475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:11.527494 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:47:11.527598 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:11.527602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:11.527678 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:11.527763 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:11.529118 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.529803 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.531250 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:11.531640 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:11.531739 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:11.536216 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:11.536232 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:11.536249 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:11.536302 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:11.539241 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:11.539863 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:11.539870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:11.539872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:11.539996 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:11.540305 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:11.540308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:11.540310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:11.540637 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:11.540849 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:11.541096 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 11:47:11.541271 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:47:11.542233 2124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:11.631166 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 1939
I20260808 11:47:11.658854 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 1423
I20260808 11:47:11.660043 1517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:11.768822 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 1423
I20260808 11:47:11.802109 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36995
--webserver_interface=127.13.158.126
--webserver_port=35147
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:36995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:12.138079 2215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:12.138414 2215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:12.138567 2215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:12.145648 2215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:12.145784 2215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:12.145889 2215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:12.146003 2215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:12.146085 2215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:12.155674 2215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:36995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=35147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:12.157007 2215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:12.158262 2215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:12.183406 2224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:12.183377 2225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:12.183360 2229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:12.184527 2215 server_base.cc:1034] running on GCE node
I20260808 11:47:12.185304 2215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:12.186889 2215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:12.188123 2215 hybrid_clock.cc:648] HybridClock initialized: now 1786189632188068 us; error 44 us; skew 500 ppm
I20260808 11:47:12.188539 2215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:12 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:12.191938 2215 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:47:12.193585 2215 webserver.cc:460] Webserver started at http://127.13.158.126:35147/ using document root <none> and password file <none>
I20260808 11:47:12.194367 2215 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:12.194535 2215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:12.200470 2215 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 11:47:12.203387 2241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:12.204658 2215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:12.204849 2215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c453528a38e7477cb71a9906695d5e3a"
format_stamp: "Formatted at 2026-08-08 11:47:08 on dist-test-slave-39l8"
server_key: "96e5afa9fb577950669d2c90b335a90e"
server_key_iv: "7bca8ae3a0ff4e0ba110d3df1736a60a"
server_key_version: "encryptionkey@0"
I20260808 11:47:12.205871 2215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:12.224014 2215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:12.228056 2215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:12.228364 2215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:12.247128 2215 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:36995
I20260808 11:47:12.247151 2332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:36995 every 8 connection(s)
I20260808 11:47:12.248684 2215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:47:12.255355 2333 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:47:12.258479 2333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Bootstrap starting.
I20260808 11:47:12.258992 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 2215
I20260808 11:47:12.259783 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:34571
--local_ip_for_outbound_sockets=127.13.158.65
--tserver_master_addrs=127.13.158.126:36995
--webserver_port=34567
--webserver_interface=127.13.158.65
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:47:12.278230 2333 log.cc:824] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:12.290195 2333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:47:12.290807 2333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Bootstrap complete.
I20260808 11:47:12.299971 2333 raft_consensus.cc:348] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:12.300876 2333 raft_consensus.cc:729] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c453528a38e7477cb71a9906695d5e3a, State: Initialized, Role: FOLLOWER
I20260808 11:47:12.301463 2333 consensus_queue.cc:260] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:12.301714 2333 raft_consensus.cc:388] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:47:12.301851 2333 raft_consensus.cc:482] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:47:12.302026 2333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:47:12.305688 2333 raft_consensus.cc:504] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:12.306190 2333 leader_election.cc:302] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c453528a38e7477cb71a9906695d5e3a; no voters:
I20260808 11:47:12.306897 2333 leader_election.cc:288] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 11:47:12.307039 2337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:47:12.308274 2337 raft_consensus.cc:686] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 2 LEADER]: Becoming Leader. State: Replica: c453528a38e7477cb71a9906695d5e3a, State: Running, Role: LEADER
I20260808 11:47:12.308926 2337 consensus_queue.cc:237] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:12.309329 2333 sys_catalog.cc:564] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:47:12.311914 2339 sys_catalog.cc:455] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c453528a38e7477cb71a9906695d5e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } } }
I20260808 11:47:12.311911 2340 sys_catalog.cc:455] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c453528a38e7477cb71a9906695d5e3a. Latest consensus state: current_term: 2 leader_uuid: "c453528a38e7477cb71a9906695d5e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } } }
I20260808 11:47:12.312356 2339 sys_catalog.cc:458] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:12.312409 2340 sys_catalog.cc:458] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:12.318296 2343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:47:12.327452 2343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:47:12.329296 2343 catalog_manager.cc:1259] Loaded cluster ID: 446933dd9175472f802578c280d2b8d1
I20260808 11:47:12.329454 2343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:47:12.333431 2343 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 11:47:12.357474 2343 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 11:47:12.368302 2343 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:47:12.370929 2343 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Loaded TSK: 0
I20260808 11:47:12.373706 2343 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 11:47:12.627518 2335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:12.627800 2335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:12.627964 2335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:12.635447 2335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:12.635638 2335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:12.635828 2335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:47:12.645643 2335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:34571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=34567
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:12.647114 2335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:12.648444 2335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:12.663455 2373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:12.663499 2377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:12.663841 2335 server_base.cc:1034] running on GCE node
W20260808 11:47:12.664163 2372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:12.664827 2335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:12.665868 2335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:12.667122 2335 hybrid_clock.cc:648] HybridClock initialized: now 1786189632667063 us; error 47 us; skew 500 ppm
I20260808 11:47:12.667503 2335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:12 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:12.670848 2335 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:47:12.672396 2335 webserver.cc:460] Webserver started at http://127.13.158.65:34567/ using document root <none> and password file <none>
I20260808 11:47:12.673085 2335 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:12.673264 2335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:12.678642 2335 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:12.681771 2385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:12.683235 2335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:12.683410 2335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc"
format_stamp: "Formatted at 2026-08-08 11:47:09 on dist-test-slave-39l8"
server_key: "86e0053708335184a64ee4631c168174"
server_key_iv: "f58e9389d820414e1f24ec6791319563"
server_key_version: "encryptionkey@0"
I20260808 11:47:12.684347 2335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:12.701588 2335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:12.705615 2335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:12.705905 2335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:12.706971 2335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:12.708244 2335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:12.708370 2335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:12.708508 2335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:12.708599 2335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:12.742513 2335 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:34571
I20260808 11:47:12.742568 2561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:34571 every 8 connection(s)
I20260808 11:47:12.744007 2335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:47:12.748889 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 2335
I20260808 11:47:12.749645 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:44801
--local_ip_for_outbound_sockets=127.13.158.66
--tserver_master_addrs=127.13.158.126:36995
--webserver_port=45421
--webserver_interface=127.13.158.66
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:47:12 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189632, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:12.771041 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:12.746953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:51577)
0808 11:47:12.747768 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:12.747811 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:12.750222 (+ 2411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:12.751584 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:12.751596 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:12.752382 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:12.753132 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:12.753171 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:12.755364 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:12.755372 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:12.757481 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:12.757499 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:12.758597 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:12.758607 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:12.758940 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:12.759647 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:12.759686 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:12.762620 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:12.767155 (+ 4535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:12.767162 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:12.767172 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:12.767568 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:12.768053 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:12.768056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:12.768058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:12.768223 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:12.769169 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:12.769175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:12.769448 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 11:47:12.769671 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 11:47:12.770028 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":344,"threads_started":1}
I20260808 11:47:12.771569 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:12.746854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:51577 (local address 127.13.158.126:36995)
0808 11:47:12.747454 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:12.747466 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:12.747934 (+ 468us) server_negotiation.cc:407] Connection header received
0808 11:47:12.750774 (+ 2840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:12.750788 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:12.751086 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:12.751367 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:12.753380 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:12.755183 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:12.757747 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:12.758352 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:12.758489 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:12.762948 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:12.762960 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:12.762980 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:12.763035 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:12.767023 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:12.767705 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:12.767716 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:12.767726 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:12.767932 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:12.768330 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:12.768338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:12.768343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:12.769031 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:12.769245 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:12.769905 (+ 660us) server_negotiation.cc:299] Negotiation successful
0808 11:47:12.770373 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":263,"threads_started":1}
I20260808 11:47:12.774258 2562 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36995
I20260808 11:47:12.774572 2562 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:12.775532 2562 heartbeater.cc:511] Master 127.13.158.126:36995 requested a full tablet report, sending...
I20260808 11:47:12.777828 2267 ts_manager.cc:194] Registered new tserver with Master: 4d6c52a31c6e498f81c9b9ec80ad67cc (127.13.158.65:34571)
I20260808 11:47:12.779508 2267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:51577
W20260808 11:47:13.089205 2568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:13.089481 2568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:13.089571 2568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:13.096592 2568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:13.096709 2568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:13.096823 2568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:47:13.106634 2568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:44801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=45421
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:13.107879 2568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:13.109197 2568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:13.122606 2577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:13.122656 2582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:13.122747 2578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:13.123636 2568 server_base.cc:1034] running on GCE node
I20260808 11:47:13.124197 2568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:13.125066 2568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:13.126258 2568 hybrid_clock.cc:648] HybridClock initialized: now 1786189633126218 us; error 37 us; skew 500 ppm
I20260808 11:47:13.126587 2568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:13 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189633, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:13.129618 2568 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:47:13.131122 2568 webserver.cc:460] Webserver started at http://127.13.158.66:45421/ using document root <none> and password file <none>
I20260808 11:47:13.131773 2568 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:13.131934 2568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:13.137120 2568 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:47:13.139783 2592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:13.140977 2568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:13.141161 2568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "3130deefc3234804bf4185e3d47c1104"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "16225f1652b1b25db94182cc74d70325"
server_key_iv: "71753af7df8d9c23195037eaebf565b3"
server_key_version: "encryptionkey@0"
I20260808 11:47:13.142068 2568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:13.164518 2568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:13.168280 2568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:13.168550 2568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:13.169516 2568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:13.170877 2568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:13.171027 2568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:13.171170 2568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:13.171278 2568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:13.204620 2568 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:44801
I20260808 11:47:13.204711 2762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:44801 every 8 connection(s)
I20260808 11:47:13.206058 2568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:47:13.212823 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 2568
I20260808 11:47:13.213583 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:33489
--local_ip_for_outbound_sockets=127.13.158.67
--tserver_master_addrs=127.13.158.126:36995
--webserver_port=44937
--webserver_interface=127.13.158.67
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:47:13 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189633, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:13.227782 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.208582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:38763 (local address 127.13.158.126:36995)
0808 11:47:13.208762 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.208766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.209915 (+ 1149us) server_negotiation.cc:407] Connection header received
0808 11:47:13.211841 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.211847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.211933 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.212067 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:13.214080 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.214849 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.217254 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.217668 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.217757 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.221939 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:13.221944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:13.221947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:13.221979 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:13.224566 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:13.225384 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:13.225395 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:13.225397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:13.225511 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:13.225939 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:13.225944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:13.225947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:13.226294 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:13.226458 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.227187 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.227416 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:47:13.228708 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.208986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:38763)
0808 11:47:13.209764 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:13.209812 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:13.211573 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:13.212347 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:13.212360 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:13.213093 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:13.213889 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.213917 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.215003 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.215012 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.217094 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.217113 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.217916 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.217926 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.218263 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.219038 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:13.219079 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:13.221728 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:13.224705 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:13.224712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:13.224725 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:13.225261 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:13.225636 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:13.225640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:13.225644 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:13.225825 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:13.226440 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:13.226447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:13.226823 (+ 376us) client_negotiation.cc:769] Sending connection context
0808 11:47:13.227095 (+ 272us) client_negotiation.cc:240] Negotiation successful
0808 11:47:13.227601 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":294,"threads_started":1}
I20260808 11:47:13.230124 2764 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36995
I20260808 11:47:13.230422 2764 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:13.231177 2764 heartbeater.cc:511] Master 127.13.158.126:36995 requested a full tablet report, sending...
I20260808 11:47:13.232676 2267 ts_manager.cc:194] Registered new tserver with Master: 3130deefc3234804bf4185e3d47c1104 (127.13.158.66:44801)
I20260808 11:47:13.233909 2267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:38763
W20260808 11:47:13.556149 2767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:13.556448 2767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:13.556547 2767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:13.563695 2767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:13.563807 2767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:13.563931 2767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:47:13.573701 2767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:33489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=44937
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:13.574945 2767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:13.576282 2767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:13.589699 2774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:13.589741 2776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:13.589781 2780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:13.595817 2767 server_base.cc:1034] running on GCE node
I20260808 11:47:13.596375 2767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:13.597254 2767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:13.598450 2767 hybrid_clock.cc:648] HybridClock initialized: now 1786189633598389 us; error 51 us; skew 500 ppm
I20260808 11:47:13.598850 2767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:13 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189633, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:13.601914 2767 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:47:13.603389 2767 webserver.cc:460] Webserver started at http://127.13.158.67:44937/ using document root <none> and password file <none>
I20260808 11:47:13.604033 2767 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:13.604169 2767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:13.609387 2767 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:47:13.612190 2788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:13.613528 2767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:47:13.613705 2767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "6c8c11d305b540309f1baed8738c9c94"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "269ba3c5f36aa1e47a398daaa66647b2"
server_key_iv: "a35affc198cea11ddbc016931a129fb5"
server_key_version: "encryptionkey@0"
I20260808 11:47:13.614642 2767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:13.639881 2767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:13.643847 2767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:13.644145 2767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:13.645270 2767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:13.646910 2767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:13.647032 2767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:13.647173 2767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:13.647262 2767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:13.682233 2767 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:33489
I20260808 11:47:13.682322 2942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:33489 every 8 connection(s)
I20260808 11:47:13.683897 2767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:47:13.694063 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 2767
Aug 08 11:47:13 dist-test-slave-39l8 krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189633, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:13.705060 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.686427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:57933 (local address 127.13.158.126:36995)
0808 11:47:13.686630 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.686635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.687832 (+ 1197us) server_negotiation.cc:407] Connection header received
0808 11:47:13.689732 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.689736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.689805 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.689904 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:13.691316 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.691991 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.693461 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.693868 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.693955 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.698618 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:13.698624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:13.698629 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:13.698666 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:13.701731 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:13.702645 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:13.702649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:13.702651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:13.702843 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:13.703358 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:13.703361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:13.703363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:13.703619 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:13.703728 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.704673 (+ 945us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.704834 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 11:47:13.706197 2946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.686925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:57933)
0808 11:47:13.687668 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:13.687719 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:13.689491 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:13.690097 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:13.690105 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:13.690570 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:13.691169 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.691186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.692143 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.692147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.693341 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.693354 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.694091 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.694100 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.694398 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.695201 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:13.695234 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:13.698422 (+ 3188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:13.701867 (+ 3445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:13.701875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:13.701889 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:13.702521 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:13.702970 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:13.702978 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:13.702985 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:13.703230 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:13.703768 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:13.703779 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:13.704237 (+ 458us) client_negotiation.cc:769] Sending connection context
0808 11:47:13.704598 (+ 361us) client_negotiation.cc:240] Negotiation successful
0808 11:47:13.705138 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":330,"threads_started":1}
I20260808 11:47:13.707646 2944 heartbeater.cc:347] Connected to a master server at 127.13.158.126:36995
I20260808 11:47:13.707978 2944 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:13.708745 2944 heartbeater.cc:511] Master 127.13.158.126:36995 requested a full tablet report, sending...
I20260808 11:47:13.710372 2267 ts_manager.cc:194] Registered new tserver with Master: 6c8c11d305b540309f1baed8738c9c94 (127.13.158.67:33489)
I20260808 11:47:13.711288 2267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:57933
I20260808 11:47:13.711812 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.695088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.13.158.126:36995)
0808 11:47:13.695619 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.695625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.695949 (+ 324us) server_negotiation.cc:407] Connection header received
0808 11:47:13.696119 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.696123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.696219 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.696356 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:13.698366 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.699500 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.701005 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.701683 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.701819 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.705479 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:13.705483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:13.705486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:13.705513 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:13.708810 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:13.709530 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:13.709537 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:13.709541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:13.709710 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:13.710135 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:13.710140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:13.710144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:13.710430 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:13.710724 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.711201 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.711459 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":391,"thread_start_us":257,"threads_started":1}
I20260808 11:47:13.714329 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:47:13.720992 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.715363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.13.158.126:36995)
0808 11:47:13.715608 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.715615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.715909 (+ 294us) server_negotiation.cc:407] Connection header received
0808 11:47:13.716168 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.716172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.716284 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.716430 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:47:13.716839 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.717495 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.719072 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.719514 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.719610 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.720067 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:47:13.720475 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:47:13.720595 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.720673 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.720729 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 11:47:13.721837 2267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:47:13.729362 2267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:47:13.755368 2961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.741084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52960 (local address 127.13.158.65:34571)
0808 11:47:13.743248 (+ 2164us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.743260 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.743292 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:47:13.743396 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.743403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.743657 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.743915 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:13.744518 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.746306 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.750958 (+ 4652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.752932 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.753087 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.753362 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.754854 (+ 1492us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.755030 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1969,"thread_start_us":1817,"threads_started":1}
I20260808 11:47:13.755407 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.740885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:34571 (local address 127.0.0.1:52960)
0808 11:47:13.742140 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:13.742189 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:13.742393 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:13.743961 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:13.743968 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:13.743997 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:13.744332 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.744343 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.746438 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.746442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.750698 (+ 4256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.750734 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.753619 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.753625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.754581 (+ 956us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.754639 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 11:47:13.754804 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 11:47:13.755005 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"mutex_wait_us":32,"thread_start_us":2364,"threads_started":1}
I20260808 11:47:13.757313 2960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.741310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:33489 (local address 127.0.0.1:43972)
0808 11:47:13.743534 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:13.743553 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:13.743689 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:13.748416 (+ 4727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:13.748424 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:13.748452 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:13.748864 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.748881 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.751607 (+ 2726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.751616 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.754532 (+ 2916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.754554 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.756449 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.756459 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.756642 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.756683 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 11:47:13.756784 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 11:47:13.756919 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2083,"mutex_wait_us":47,"thread_start_us":6047,"threads_started":1}
I20260808 11:47:13.757339 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.747051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.13.158.67:33489)
0808 11:47:13.747836 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.747845 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.747886 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:47:13.747995 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.748002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.748197 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.749085 (+ 888us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:13.749362 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.751377 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.754697 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.756197 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.756334 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.756561 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.756846 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.757008 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":929,"threads_started":1}
I20260808 11:47:13.759433 2462 tablet_service.cc:1467] Processing CreateTablet for tablet 52ed19e5136c4da4a9c1e47d18028291 (DEFAULT_TABLE table=test-table [id=a2a6e52807e940c4ae583a9344b4e24f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:13.760723 2462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ed19e5136c4da4a9c1e47d18028291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:13.761363 2851 tablet_service.cc:1467] Processing CreateTablet for tablet 52ed19e5136c4da4a9c1e47d18028291 (DEFAULT_TABLE table=test-table [id=a2a6e52807e940c4ae583a9344b4e24f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:13.762603 2851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ed19e5136c4da4a9c1e47d18028291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:13.763267 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.747819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:44801 (local address 127.0.0.1:49710)
0808 11:47:13.754672 (+ 6853us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:13.754690 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:13.754849 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:13.755948 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:13.755953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:13.755978 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:13.756373 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.756384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.758333 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.758338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.760398 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.760414 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.762414 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.762421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.762618 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.762664 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 11:47:13.762788 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 11:47:13.762923 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6736,"thread_start_us":6561,"threads_started":1}
I20260808 11:47:13.764073 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.748081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.13.158.66:44801)
0808 11:47:13.754352 (+ 6271us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.754361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.754984 (+ 623us) server_negotiation.cc:407] Connection header received
0808 11:47:13.755099 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.755109 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.755327 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.755966 (+ 639us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:13.756578 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.758161 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.760572 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.762115 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.762270 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.762465 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.763565 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.763740 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6076,"thread_start_us":5941,"threads_started":1}
I20260808 11:47:13.767482 2669 tablet_service.cc:1467] Processing CreateTablet for tablet 52ed19e5136c4da4a9c1e47d18028291 (DEFAULT_TABLE table=test-table [id=a2a6e52807e940c4ae583a9344b4e24f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:13.768771 2669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ed19e5136c4da4a9c1e47d18028291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:13.773249 2967 tablet_bootstrap.cc:492] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Bootstrap starting.
I20260808 11:47:13.776242 2967 tablet_bootstrap.cc:654] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:13.777331 2967 log.cc:824] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:13.778100 2968 tablet_bootstrap.cc:492] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Bootstrap starting.
I20260808 11:47:13.781620 2968 tablet_bootstrap.cc:654] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:13.782166 2967 tablet_bootstrap.cc:492] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: No bootstrap required, opened a new log
I20260808 11:47:13.782428 2967 ts_tablet_manager.cc:1397] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260808 11:47:13.782883 2968 log.cc:824] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:13.784299 2562 heartbeater.cc:503] Master 127.13.158.126:36995 was elected leader, sending a full tablet report...
I20260808 11:47:13.789494 2968 tablet_bootstrap.cc:492] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: No bootstrap required, opened a new log
I20260808 11:47:13.789750 2968 ts_tablet_manager.cc:1397] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260808 11:47:13.794842 2970 tablet_bootstrap.cc:492] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Bootstrap starting.
I20260808 11:47:13.796458 2967 raft_consensus.cc:348] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.796888 2967 raft_consensus.cc:374] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:13.797000 2967 raft_consensus.cc:729] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6c52a31c6e498f81c9b9ec80ad67cc, State: Initialized, Role: FOLLOWER
I20260808 11:47:13.797788 2970 tablet_bootstrap.cc:654] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:13.797605 2967 consensus_queue.cc:260] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.799118 2970 log.cc:824] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:13.799342 2967 ts_tablet_manager.cc:1428] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260808 11:47:13.800239 2968 raft_consensus.cc:348] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.800663 2968 raft_consensus.cc:374] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:13.800765 2968 raft_consensus.cc:729] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8c11d305b540309f1baed8738c9c94, State: Initialized, Role: FOLLOWER
I20260808 11:47:13.801353 2968 consensus_queue.cc:260] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.802515 2970 tablet_bootstrap.cc:492] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: No bootstrap required, opened a new log
I20260808 11:47:13.802507 2944 heartbeater.cc:503] Master 127.13.158.126:36995 was elected leader, sending a full tablet report...
I20260808 11:47:13.802811 2970 ts_tablet_manager.cc:1397] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260808 11:47:13.803165 2968 ts_tablet_manager.cc:1428] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260808 11:47:13.810356 2970 raft_consensus.cc:348] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.810767 2970 raft_consensus.cc:374] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:13.810871 2970 raft_consensus.cc:729] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3130deefc3234804bf4185e3d47c1104, State: Initialized, Role: FOLLOWER
I20260808 11:47:13.811373 2970 consensus_queue.cc:260] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.812367 2764 heartbeater.cc:503] Master 127.13.158.126:36995 was elected leader, sending a full tablet report...
I20260808 11:47:13.812942 2970 ts_tablet_manager.cc:1428] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260808 11:47:13.893343 2972 raft_consensus.cc:482] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:47:13.893718 2972 raft_consensus.cc:504] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.895550 2972 leader_election.cc:288] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c8c11d305b540309f1baed8738c9c94 (127.13.158.67:33489), 3130deefc3234804bf4185e3d47c1104 (127.13.158.66:44801)
I20260808 11:47:13.902093 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.896076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:52623 (local address 127.13.158.67:33489)
0808 11:47:13.896286 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.896291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.896762 (+ 471us) server_negotiation.cc:407] Connection header received
0808 11:47:13.896984 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.896993 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.897077 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.897176 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:13.897877 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.898609 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.900403 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.901146 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.901331 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.901406 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.901676 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.901799 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:47:13.902213 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.895893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:33489 (local address 127.13.158.65:52623)
0808 11:47:13.896650 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:13.896671 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:13.896816 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:13.897304 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:13.897310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:13.897345 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:13.897711 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.897720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.898786 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.898794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.900240 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.900253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.901353 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.901359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.901511 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.901535 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:47:13.901641 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 11:47:13.901766 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":21,"thread_start_us":395,"threads_started":1}
I20260808 11:47:13.902520 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.896958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:36825 (local address 127.13.158.66:44801)
0808 11:47:13.897198 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.897204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.897532 (+ 328us) server_negotiation.cc:407] Connection header received
0808 11:47:13.897746 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.897751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.897809 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.897905 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:13.898429 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.899183 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.900788 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.901514 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.901694 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.901780 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.902090 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.902268 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 11:47:13.902732 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.896764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:44801 (local address 127.13.158.65:36825)
0808 11:47:13.897412 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:13.897429 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:13.897578 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:13.897919 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:13.897923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:13.897945 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:13.898278 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.898286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.899318 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.899323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.900632 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:13.900644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.901739 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.901745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:13.901900 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.901923 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:47:13.902023 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 11:47:13.902184 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":1,"thread_start_us":350,"threads_started":1}
I20260808 11:47:13.903533 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ed19e5136c4da4a9c1e47d18028291" candidate_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8c11d305b540309f1baed8738c9c94" is_pre_election: true
I20260808 11:47:13.903919 2700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ed19e5136c4da4a9c1e47d18028291" candidate_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3130deefc3234804bf4185e3d47c1104" is_pre_election: true
I20260808 11:47:13.904034 2879 raft_consensus.cc:2454] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6c52a31c6e498f81c9b9ec80ad67cc in term 0.
I20260808 11:47:13.904345 2700 raft_consensus.cc:2454] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d6c52a31c6e498f81c9b9ec80ad67cc in term 0.
I20260808 11:47:13.904716 2389 leader_election.cc:302] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6c52a31c6e498f81c9b9ec80ad67cc, 6c8c11d305b540309f1baed8738c9c94; no voters:
I20260808 11:47:13.905078 2972 raft_consensus.cc:2781] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:47:13.905223 2972 raft_consensus.cc:482] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:47:13.905347 2972 raft_consensus.cc:3037] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:13.907598 2972 raft_consensus.cc:504] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.908151 2972 leader_election.cc:288] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [CANDIDATE]: Term 1 election: Requested vote from peers 6c8c11d305b540309f1baed8738c9c94 (127.13.158.67:33489), 3130deefc3234804bf4185e3d47c1104 (127.13.158.66:44801)
I20260808 11:47:13.908708 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ed19e5136c4da4a9c1e47d18028291" candidate_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8c11d305b540309f1baed8738c9c94"
I20260808 11:47:13.908756 2700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ed19e5136c4da4a9c1e47d18028291" candidate_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3130deefc3234804bf4185e3d47c1104"
I20260808 11:47:13.908950 2879 raft_consensus.cc:3037] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:13.909081 2700 raft_consensus.cc:3037] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:13.911563 2879 raft_consensus.cc:2454] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6c52a31c6e498f81c9b9ec80ad67cc in term 1.
I20260808 11:47:13.911567 2700 raft_consensus.cc:2454] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d6c52a31c6e498f81c9b9ec80ad67cc in term 1.
I20260808 11:47:13.911979 2389 leader_election.cc:302] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6c52a31c6e498f81c9b9ec80ad67cc, 6c8c11d305b540309f1baed8738c9c94; no voters:
I20260808 11:47:13.912299 2972 raft_consensus.cc:2781] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:47:13.912786 2972 raft_consensus.cc:686] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 LEADER]: Becoming Leader. State: Replica: 4d6c52a31c6e498f81c9b9ec80ad67cc, State: Running, Role: LEADER
I20260808 11:47:13.913249 2972 consensus_queue.cc:237] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } }
I20260808 11:47:13.918188 2267 catalog_manager.cc:5495] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d6c52a31c6e498f81c9b9ec80ad67cc (127.13.158.65). New cstate: current_term: 1 leader_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 34571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 } health_report { overall_health: UNKNOWN } } }
W20260808 11:47:13.935647 2945 tablet.cc:2369] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:13.941571 2961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.935970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52966 (local address 127.13.158.65:34571)
0808 11:47:13.936160 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.936166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.936183 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:47:13.936396 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.936400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.936477 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.936649 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:47:13.937029 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.937798 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.939347 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.939786 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.939927 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.940472 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:47:13.940972 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:47:13.941127 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:13.941215 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 11:47:13.941298 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 11:47:13.949774 2700 raft_consensus.cc:1261] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 1 FOLLOWER]: Refusing update from remote peer 4d6c52a31c6e498f81c9b9ec80ad67cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:13.949780 2879 raft_consensus.cc:1261] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 1 FOLLOWER]: Refusing update from remote peer 4d6c52a31c6e498f81c9b9ec80ad67cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:13.950798 2972 consensus_queue.cc:1035] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3130deefc3234804bf4185e3d47c1104" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:47:13.951249 2981 consensus_queue.cc:1035] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8c11d305b540309f1baed8738c9c94" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 33489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:47:13.958842 2765 tablet.cc:2369] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:13.964332 2985 mvcc.cc:205] Tried to move back new op lower bound from 7316232740648824832 to 7316232740512526336. Current Snapshot: MvccSnapshot[applied={T|T < 7316232740648824832}]
I20260808 11:47:13.980950 2267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:47:13.981240 2267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:47:13.984815 2267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Sending DeleteTablet for 3 replicas of tablet 52ed19e5136c4da4a9c1e47d18028291
I20260808 11:47:13.985828 2462 tablet_service.cc:1514] Processing DeleteTablet for tablet 52ed19e5136c4da4a9c1e47d18028291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:13 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:52960
I20260808 11:47:13.986179 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 2335
I20260808 11:47:13.986099 2851 tablet_service.cc:1514] Processing DeleteTablet for tablet 52ed19e5136c4da4a9c1e47d18028291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:13 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:43972
I20260808 11:47:13.986963 2669 tablet_service.cc:1514] Processing DeleteTablet for tablet 52ed19e5136c4da4a9c1e47d18028291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:13 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:49710
I20260808 11:47:13.987882 2998 tablet_replica.cc:330] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: stopping tablet replica
I20260808 11:47:13.988359 2998 raft_consensus.cc:2229] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:47:13.988916 2998 raft_consensus.cc:2258] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:13.989750 2997 tablet_replica.cc:330] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: stopping tablet replica
I20260808 11:47:13.990007 2997 raft_consensus.cc:2229] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:47:13.990459 2997 raft_consensus.cc:2258] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:13.993750 2998 ts_tablet_manager.cc:1905] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:14.001132 2999 tablet_replica.cc:330] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: stopping tablet replica
I20260808 11:47:14.001399 2998 ts_tablet_manager.cc:1918] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:14.001564 2998 log.cc:1196] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/52ed19e5136c4da4a9c1e47d18028291
I20260808 11:47:14.001924 2999 raft_consensus.cc:2229] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:47:14.002245 2998 ts_tablet_manager.cc:1939] T 52ed19e5136c4da4a9c1e47d18028291 P 6c8c11d305b540309f1baed8738c9c94: Deleting consensus metadata
I20260808 11:47:14.002549 2999 raft_consensus.cc:2258] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:14.004688 2997 ts_tablet_manager.cc:1905] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:14.005388 2999 ts_tablet_manager.cc:1905] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:14.007179 2961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:13.988081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.13.158.65:34571)
0808 11:47:13.988259 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:13.988266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:13.988284 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:47:13.988335 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:13.988340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:13.988417 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:13.988510 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:13.990852 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.991804 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.993860 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:13.994494 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:13.994615 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:13.998409 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:13.998424 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:13.998441 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:13.998486 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:14.002890 (+ 4404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:14.003781 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:14.003788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:14.003792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:14.003955 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:14.004461 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:14.004466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:14.004469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:14.004976 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:14.005306 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:14.006479 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0808 11:47:14.006780 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:47:14.008164 2556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:14.009744 2244 catalog_manager.cc:4841] TS 6c8c11d305b540309f1baed8738c9c94 (127.13.158.67:33489): tablet 52ed19e5136c4da4a9c1e47d18028291 (table test-table [id=a2a6e52807e940c4ae583a9344b4e24f]) successfully deleted
I20260808 11:47:14.013823 2999 ts_tablet_manager.cc:1918] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:14.013993 2999 log.cc:1196] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/52ed19e5136c4da4a9c1e47d18028291
I20260808 11:47:14.014566 2999 ts_tablet_manager.cc:1939] T 52ed19e5136c4da4a9c1e47d18028291 P 3130deefc3234804bf4185e3d47c1104: Deleting consensus metadata
I20260808 11:47:14.016652 2244 catalog_manager.cc:4841] TS 3130deefc3234804bf4185e3d47c1104 (127.13.158.66:44801): tablet 52ed19e5136c4da4a9c1e47d18028291 (table test-table [id=a2a6e52807e940c4ae583a9344b4e24f]) successfully deleted
I20260808 11:47:14.115083 2997 ts_tablet_manager.cc:1918] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:14.115314 2997 log.cc:1196] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/52ed19e5136c4da4a9c1e47d18028291
I20260808 11:47:14.115852 2997 ts_tablet_manager.cc:1939] T 52ed19e5136c4da4a9c1e47d18028291 P 4d6c52a31c6e498f81c9b9ec80ad67cc: Deleting consensus metadata
I20260808 11:47:14.117532 2242 catalog_manager.cc:4841] TS 4d6c52a31c6e498f81c9b9ec80ad67cc (127.13.158.65:34571): tablet 52ed19e5136c4da4a9c1e47d18028291 (table test-table [id=a2a6e52807e940c4ae583a9344b4e24f]) successfully deleted
I20260808 11:47:14.118067 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 2335
I20260808 11:47:14.142354 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 2568
I20260808 11:47:14.155426 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:14.143135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.13.158.66:44801)
0808 11:47:14.143322 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:14.143326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:14.143344 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:47:14.143461 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:14.143464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:14.143534 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:14.143616 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:14.144969 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:14.145749 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:14.147322 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:14.147726 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:14.147833 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:14.150269 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:14.150279 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:14.150310 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:14.150339 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:14.152930 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:14.153613 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:14.153619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:14.153621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:14.153743 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:14.154049 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:14.154051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:14.154053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:14.154389 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:14.154625 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:14.154918 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 11:47:14.155088 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 11:47:14.156113 2757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:14.245846 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 2568
I20260808 11:47:14.277197 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 2767
I20260808 11:47:14.290675 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:14.278034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.13.158.67:33489)
0808 11:47:14.278292 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:14.278297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:14.278318 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:47:14.278477 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:14.278483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:14.278556 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:14.278693 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:14.280124 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:14.281220 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:14.282466 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:14.283096 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:14.283208 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:14.285695 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:14.285719 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:14.285735 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:14.285766 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:14.288368 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:14.288990 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:14.288997 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:14.289000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:14.289133 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:14.289453 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:14.289455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:14.289457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:14.289779 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:14.289987 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:14.290260 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 11:47:14.290414 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260808 11:47:14.291366 2938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:14.379282 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 2767
I20260808 11:47:14.404433 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 2215
I20260808 11:47:14.405633 2326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:14.516654 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 2215
I20260808 11:47:14.546271 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36995
--webserver_interface=127.13.158.126
--webserver_port=35147
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:36995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:14.882305 3007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:14.882575 3007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:14.882673 3007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:14.889601 3007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:14.889696 3007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:14.889741 3007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:14.889792 3007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:14.889835 3007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:14.899158 3007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:36995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=35147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:14.900355 3007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:14.901513 3007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:14.913424 3013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:14.913458 3020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:14.913378 3016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:14.914252 3007 server_base.cc:1034] running on GCE node
I20260808 11:47:14.915050 3007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:14.916301 3007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:14.917531 3007 hybrid_clock.cc:648] HybridClock initialized: now 1786189634917453 us; error 55 us; skew 500 ppm
I20260808 11:47:14.918001 3007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:14 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189634, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:14.925004 3007 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:47:14.926403 3007 webserver.cc:460] Webserver started at http://127.13.158.126:35147/ using document root <none> and password file <none>
I20260808 11:47:14.927188 3007 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:14.927328 3007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:14.932717 3007 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 11:47:14.935441 3029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:14.936697 3007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:14.936861 3007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c453528a38e7477cb71a9906695d5e3a"
format_stamp: "Formatted at 2026-08-08 11:47:08 on dist-test-slave-39l8"
server_key: "96e5afa9fb577950669d2c90b335a90e"
server_key_iv: "7bca8ae3a0ff4e0ba110d3df1736a60a"
server_key_version: "encryptionkey@0"
I20260808 11:47:14.937846 3007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:14.959867 3007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:14.963923 3007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:14.964215 3007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:14.982924 3103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:36995 every 8 connection(s)
I20260808 11:47:14.982949 3007 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:36995
I20260808 11:47:14.984416 3007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:47:14.990834 3105 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:47:14.992403 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 3007
I20260808 11:47:14.993450 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:34571
--local_ip_for_outbound_sockets=127.13.158.65
--tserver_master_addrs=127.13.158.126:36995
--webserver_port=34567
--webserver_interface=127.13.158.65
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:47:14.995265 3105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Bootstrap starting.
I20260808 11:47:15.018277 3105 log.cc:824] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:15.033893 3105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:47:15.034436 3105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a: Bootstrap complete.
I20260808 11:47:15.043869 3105 raft_consensus.cc:348] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:15.044734 3105 raft_consensus.cc:729] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c453528a38e7477cb71a9906695d5e3a, State: Initialized, Role: FOLLOWER
I20260808 11:47:15.045276 3105 consensus_queue.cc:260] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:15.045531 3105 raft_consensus.cc:388] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:47:15.045670 3105 raft_consensus.cc:482] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:47:15.045841 3105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 2 FOLLOWER]: Advancing to term 3
I20260808 11:47:15.049562 3105 raft_consensus.cc:504] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:15.050028 3105 leader_election.cc:302] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c453528a38e7477cb71a9906695d5e3a; no voters:
I20260808 11:47:15.050740 3105 leader_election.cc:288] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 11:47:15.050858 3110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 3 FOLLOWER]: Leader election won for term 3
I20260808 11:47:15.052233 3110 raft_consensus.cc:686] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [term 3 LEADER]: Becoming Leader. State: Replica: c453528a38e7477cb71a9906695d5e3a, State: Running, Role: LEADER
I20260808 11:47:15.052767 3110 consensus_queue.cc:237] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } }
I20260808 11:47:15.053128 3105 sys_catalog.cc:564] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:47:15.055568 3112 sys_catalog.cc:455] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c453528a38e7477cb71a9906695d5e3a. Latest consensus state: current_term: 3 leader_uuid: "c453528a38e7477cb71a9906695d5e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } } }
I20260808 11:47:15.055998 3112 sys_catalog.cc:458] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:15.056572 3111 sys_catalog.cc:455] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c453528a38e7477cb71a9906695d5e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c453528a38e7477cb71a9906695d5e3a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36995 } } }
I20260808 11:47:15.056950 3111 sys_catalog.cc:458] T 00000000000000000000000000000000 P c453528a38e7477cb71a9906695d5e3a [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:15.061190 3120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:47:15.073473 3120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:47:15.074987 3120 catalog_manager.cc:1259] Loaded cluster ID: 446933dd9175472f802578c280d2b8d1
I20260808 11:47:15.075083 3120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 11:47:15.077320 3120 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ffb862219ad google::LogMessage::Fail() at ??:0
@ 0x7ffb8622488a google::LogMessage::SendToLog() at ??:0
@ 0x7ffb862214de google::LogMessage::Flush() at ??:0
@ 0x7ffb86222a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ffb9bc5a512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7ffb9bc57976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ffb9bcdf610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7ffb9bcdf439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ffb88e6d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ffb88e7091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7ffb88e70799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ffb88e4beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7ffb8c1256db start_thread at ??:0
@ 0x7ffb850bb61f clone at ??:0
W20260808 11:47:15.419472 3107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:15.419742 3107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:15.419840 3107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:15.426883 3107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:15.426978 3107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:15.427088 3107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:47:15.436394 3107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:34571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=34567
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:15.437541 3107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:15.438791 3107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:15.452252 3164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:15.452279 3166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:15.452389 3169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:15.452921 3107 server_base.cc:1034] running on GCE node
I20260808 11:47:15.453634 3107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:15.454556 3107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:15.455745 3107 hybrid_clock.cc:648] HybridClock initialized: now 1786189635455709 us; error 69 us; skew 500 ppm
I20260808 11:47:15.456116 3107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:15 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189635, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:15.459592 3107 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:47:15.460951 3107 webserver.cc:460] Webserver started at http://127.13.158.65:34567/ using document root <none> and password file <none>
I20260808 11:47:15.461617 3107 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:15.461778 3107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:15.466893 3107 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:15.471866 3179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:15.473146 3107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:47:15.473347 3107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "4d6c52a31c6e498f81c9b9ec80ad67cc"
format_stamp: "Formatted at 2026-08-08 11:47:09 on dist-test-slave-39l8"
server_key: "86e0053708335184a64ee4631c168174"
server_key_iv: "f58e9389d820414e1f24ec6791319563"
server_key_version: "encryptionkey@0"
I20260808 11:47:15.474306 3107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:15.496804 3107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:15.501104 3107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:15.501384 3107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:15.502378 3107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:15.503741 3107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:15.503870 3107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:15.504015 3107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:15.504104 3107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:15.539079 3107 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:34571
I20260808 11:47:15.539162 3361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:34571 every 8 connection(s)
I20260808 11:47:15.540521 3107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:47:15.545250 3366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:15.543568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:58273)
0808 11:47:15.544268 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:15.544478 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":265,"threads_started":1}
I20260808 11:47:15.545989 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 3107
W20260808 11:47:15.546005 3362 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:36995 (0 consecutive failures): Network error: Failed to ping master at 127.13.158.126:36995: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
I20260808 11:47:15.546797 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:44801
--local_ip_for_outbound_sockets=127.13.158.66
--tserver_master_addrs=127.13.158.126:36995
--webserver_port=45421
--webserver_interface=127.13.158.66
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:47:15.546912 3366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:15.546457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:39341)
0808 11:47:15.546661 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:15.546774 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 11:47:15.547796 3366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:15.547476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:38313)
0808 11:47:15.547609 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:15.547684 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 11:47:15.548111 3362 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:47:15.876298 3367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:15.876555 3367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:15.876763 3367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:15.883912 3367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:15.884037 3367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:15.884227 3367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:47:15.893373 3367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:44801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=45421
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:15.894536 3367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:15.895846 3367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:15.908735 3378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:15.908756 3379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:15.908818 3381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:15.909699 3367 server_base.cc:1034] running on GCE node
I20260808 11:47:15.910223 3367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:15.911043 3367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:15.912245 3367 hybrid_clock.cc:648] HybridClock initialized: now 1786189635912214 us; error 40 us; skew 500 ppm
I20260808 11:47:15.912554 3367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:15 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189635, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:15.915557 3367 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:47:15.916805 3367 webserver.cc:460] Webserver started at http://127.13.158.66:45421/ using document root <none> and password file <none>
I20260808 11:47:15.917404 3367 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:15.917503 3367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:15.922415 3367 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:47:15.925041 3392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:15.926302 3367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:47:15.926486 3367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "3130deefc3234804bf4185e3d47c1104"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "16225f1652b1b25db94182cc74d70325"
server_key_iv: "71753af7df8d9c23195037eaebf565b3"
server_key_version: "encryptionkey@0"
I20260808 11:47:15.927507 3367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:15.941934 3367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:15.945837 3367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:15.946123 3367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:15.947152 3367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:15.948426 3367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:15.948553 3367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:15.948683 3367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:15.948782 3367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:15.981631 3367 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:44801
I20260808 11:47:15.981719 3580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:44801 every 8 connection(s)
I20260808 11:47:15.983197 3367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:47:15.987846 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:15.985959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:36699)
0808 11:47:15.986616 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:15.986919 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":257,"threads_started":1}
W20260808 11:47:15.988442 3583 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:36995 (0 consecutive failures): Network error: Failed to ping master at 127.13.158.126:36995: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
I20260808 11:47:15.989166 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:15.988878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:43659)
0808 11:47:15.989004 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:15.989066 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 11:47:15.989991 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:15.989672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:42755)
0808 11:47:15.989808 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:15.989888 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 11:47:15.990242 3583 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:47:15.993428 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 3367
I20260808 11:47:15.994063 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:33489
--local_ip_for_outbound_sockets=127.13.158.67
--tserver_master_addrs=127.13.158.126:36995
--webserver_port=44937
--webserver_interface=127.13.158.67
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:16.332150 3588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:16.332413 3588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:16.332618 3588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:16.339821 3588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:16.339949 3588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:16.340143 3588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:47:16.349774 3588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:33489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=44937
--tserver_master_addrs=127.13.158.126:36995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:16.351086 3588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:16.352399 3588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:16.365499 3598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:16.365566 3594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:16.365542 3595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:16.366851 3588 server_base.cc:1034] running on GCE node
I20260808 11:47:16.367516 3588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:16.368700 3588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:16.369973 3588 hybrid_clock.cc:648] HybridClock initialized: now 1786189636369920 us; error 36 us; skew 500 ppm
I20260808 11:47:16.370318 3588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:16 dist-test-slave-39l8 krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189636, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:16.373478 3588 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:47:16.374845 3588 webserver.cc:460] Webserver started at http://127.13.158.67:44937/ using document root <none> and password file <none>
I20260808 11:47:16.375474 3588 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:16.375586 3588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:16.380898 3588 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 11:47:16.383518 3608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:16.384790 3588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:16.384958 3588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "6c8c11d305b540309f1baed8738c9c94"
format_stamp: "Formatted at 2026-08-08 11:47:10 on dist-test-slave-39l8"
server_key: "269ba3c5f36aa1e47a398daaa66647b2"
server_key_iv: "a35affc198cea11ddbc016931a129fb5"
server_key_version: "encryptionkey@0"
I20260808 11:47:16.385897 3588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:16.418622 3588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:16.422303 3588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:16.422590 3588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:16.423523 3588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:16.424834 3588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:16.424957 3588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:16.425091 3588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:16.425185 3588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:16.456714 3588 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:33489
I20260808 11:47:16.456799 3782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:33489 every 8 connection(s)
I20260808 11:47:16.458122 3588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:47:16.462855 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:16.460991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:53539)
0808 11:47:16.461725 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:16.461929 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":303,"threads_started":1}
W20260808 11:47:16.463459 3783 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:36995 (0 consecutive failures): Network error: Failed to ping master at 127.13.158.126:36995: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
I20260808 11:47:16.464309 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:16.463910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:53421)
0808 11:47:16.464080 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:16.464167 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 11:47:16.464280 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 3588
I20260808 11:47:16.465204 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:16.464850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:33953)
0808 11:47:16.464989 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:16.465068 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:47:16.465459 3783 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:47:16.549353 3787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:16.548586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:41919)
0808 11:47:16.549013 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:16.549134 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":220,"threads_started":1}
I20260808 11:47:16.991559 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:16.990772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:51809)
0808 11:47:16.991196 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:16.991332 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":208,"threads_started":1}
I20260808 11:47:17.466775 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:17.465943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:59857)
0808 11:47:17.466390 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:17.466498 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":251,"threads_started":1}
I20260808 11:47:17.551281 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:17.550478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:43165)
0808 11:47:17.550980 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:17.551102 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":286,"threads_started":1}
I20260808 11:47:17.993185 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:17.992433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:55473)
0808 11:47:17.992935 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:17.993022 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":251,"threads_started":1}
I20260808 11:47:18.468420 3793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:18.467723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:47949)
0808 11:47:18.468178 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:18.468256 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":240,"threads_started":1}
I20260808 11:47:18.552822 3794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:18.552154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:42101)
0808 11:47:18.552560 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:18.552639 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":221,"threads_started":1}
I20260808 11:47:18.995012 3795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:18.994119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:47273)
0808 11:47:18.994614 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:18.994747 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":275,"threads_started":1}
I20260808 11:47:19.470213 3797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:19.469443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:39065)
0808 11:47:19.469950 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:19.470034 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":298,"threads_started":1}
I20260808 11:47:19.554324 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:19.553633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:60233)
0808 11:47:19.554050 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:19.554130 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":207,"threads_started":1}
I20260808 11:47:19.996546 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:19.995847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:45263)
0808 11:47:19.996268 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:19.996358 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":210,"threads_started":1}
I20260808 11:47:20.471913 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:20.471246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:51603)
0808 11:47:20.471663 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:20.471748 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":218,"threads_started":1}
I20260808 11:47:20.555977 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:20.555208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:48145)
0808 11:47:20.555737 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:20.555815 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":312,"threads_started":1}
I20260808 11:47:20.998169 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:20.997454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:35967)
0808 11:47:20.997933 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:20.998009 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":261,"threads_started":1}
I20260808 11:47:21.473397 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:21.472748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:48845)
0808 11:47:21.473158 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:21.473234 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":207,"threads_started":1}
I20260808 11:47:21.557473 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:21.556780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:43243)
0808 11:47:21.557231 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:21.557316 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":233,"threads_started":1}
I20260808 11:47:21.999697 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:21.999016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:55139)
0808 11:47:21.999444 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:21.999531 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":232,"threads_started":1}
I20260808 11:47:22.475221 3808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:22.474485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:44751)
0808 11:47:22.474970 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:22.475055 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":274,"threads_started":1}
I20260808 11:47:22.559089 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:22.558296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:49603)
0808 11:47:22.558804 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:22.558883 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":215,"threads_started":1}
I20260808 11:47:23.001397 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:23.000576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:34609)
0808 11:47:23.001096 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:23.001185 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":222,"threads_started":1}
I20260808 11:47:23.476873 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:23.476061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:46681)
0808 11:47:23.476543 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:23.476629 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":230,"threads_started":1}
I20260808 11:47:23.561266 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:23.560173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:42495)
0808 11:47:23.560930 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:23.561040 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":388,"threads_started":1}
I20260808 11:47:24.003016 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:24.002243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:47173)
0808 11:47:24.002765 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:24.002851 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":281,"threads_started":1}
I20260808 11:47:24.478569 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:24.477788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:41695)
0808 11:47:24.478263 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:24.478399 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":224,"threads_started":1}
I20260808 11:47:24.562939 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:24.562120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:50693)
0808 11:47:24.562613 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:24.562692 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":259,"threads_started":1}
I20260808 11:47:25.004694 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:25.003858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:45825)
0808 11:47:25.004401 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:25.004511 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":234,"threads_started":1}
I20260808 11:47:25.480254 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:25.479472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:59767)
0808 11:47:25.479981 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:25.480084 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":258,"threads_started":1}
I20260808 11:47:25.564574 3824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:25.563896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:51185)
0808 11:47:25.564339 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:25.564418 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":251,"threads_started":1}
I20260808 11:47:26.006338 3825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:26.005649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:55201)
0808 11:47:26.006088 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:26.006180 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":248,"threads_started":1}
I20260808 11:47:26.481917 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:26.481174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:41953)
0808 11:47:26.481672 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:26.481756 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":279,"threads_started":1}
I20260808 11:47:26.566381 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:26.565683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:34891)
0808 11:47:26.566106 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:26.566197 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":230,"threads_started":1}
I20260808 11:47:27.008134 3829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:27.007436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:58825)
0808 11:47:27.007868 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:27.007959 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":230,"threads_started":1}
I20260808 11:47:27.483572 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:27.482912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:52659)
0808 11:47:27.483316 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:27.483403 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":205,"threads_started":1}
I20260808 11:47:27.568189 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:27.567465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:57369)
0808 11:47:27.567934 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:27.568024 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":215,"threads_started":1}
I20260808 11:47:28.009788 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:28.009035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:41197)
0808 11:47:28.009528 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:28.009606 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":291,"threads_started":1}
I20260808 11:47:28.485162 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:28.484479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:42139)
0808 11:47:28.484919 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:28.485012 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":202,"threads_started":1}
I20260808 11:47:28.569891 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:28.569071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:36003)
0808 11:47:28.569614 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:28.569705 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":310,"threads_started":1}
I20260808 11:47:29.011688 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:29.010800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:54577)
0808 11:47:29.011358 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:29.011451 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":280,"threads_started":1}
I20260808 11:47:29.486976 3841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:29.486187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:49423)
0808 11:47:29.486640 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:29.486790 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":233,"threads_started":1}
I20260808 11:47:29.571530 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:29.570735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:52929)
0808 11:47:29.571275 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:29.571357 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":255,"threads_started":1}
I20260808 11:47:30.013825 3844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:30.012798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:36969)
0808 11:47:30.013384 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:30.013515 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":304,"threads_started":1}
I20260808 11:47:30.488581 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:30.487860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:58033)
0808 11:47:30.488317 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:30.488399 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":254,"threads_started":1}
I20260808 11:47:30.573726 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:30.572911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:40525)
0808 11:47:30.573423 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:30.573535 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":314,"threads_started":1}
I20260808 11:47:31.015764 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:31.015006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.66:41887)
0808 11:47:31.015474 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:31.015574 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":244,"threads_started":1}
I20260808 11:47:31.476028 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 3107
I20260808 11:47:31.490545 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:31.489715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.67:36449)
0808 11:47:31.490256 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:31.490356 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":267,"threads_started":1}
I20260808 11:47:31.494504 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:31.477120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37026 (local address 127.13.158.65:34571)
0808 11:47:31.477784 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:31.477799 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:31.477957 (+ 158us) server_negotiation.cc:407] Connection header received
0808 11:47:31.479938 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:31.479948 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:31.480397 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:31.480688 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:31.482182 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:31.483502 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:31.484745 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:31.485389 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:31.485520 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:31.488241 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:31.488249 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:31.488264 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:31.488299 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:31.491575 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:31.492225 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:31.492231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:31.492237 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:31.492393 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:31.492716 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:31.492719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:31.492721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:31.493218 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:31.493430 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:31.493821 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 11:47:31.494234 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":298,"threads_started":1}
I20260808 11:47:31.495921 3355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:31.580928 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:31.579768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:36995 (local address 127.13.158.65:52125)
0808 11:47:31.580361 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:31.580627 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":267,"threads_started":1}
I20260808 11:47:31.582911 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 3107
I20260808 11:47:31.608366 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 3367
I20260808 11:47:31.625463 3858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:31.609346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.13.158.66:44801)
0808 11:47:31.609942 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:31.609955 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:31.609994 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:47:31.611715 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:31.611725 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:31.612173 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:31.612467 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:31.613796 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:31.615122 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:31.616293 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:31.616903 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:31.617028 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:31.619762 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:31.619770 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:31.619784 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:31.619819 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:31.622564 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:31.623296 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:31.623303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:31.623309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:31.623475 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:31.623796 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:31.623799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:31.623801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:31.624282 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:31.624501 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:31.624852 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 11:47:31.625182 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":301,"threads_started":1}
I20260808 11:47:31.626924 3574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:31.710461 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 3367
I20260808 11:47:31.735526 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 3588
I20260808 11:47:31.753999 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:31.736604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.13.158.67:33489)
0808 11:47:31.737260 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:31.737274 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:31.737315 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:47:31.739086 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:31.739098 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:31.739561 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:31.739895 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:31.741449 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:31.742665 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:31.744104 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:31.744794 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:31.745016 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:31.747914 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:31.747922 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:31.747937 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:31.747974 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:31.750777 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:31.751594 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:31.751602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:31.751608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:31.751801 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:31.752169 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:31.752173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:31.752175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:31.752731 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:31.752967 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:31.753331 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 11:47:31.753699 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":307,"threads_started":1}
I20260808 11:47:31.755453 3777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:31.839476 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 3588
2026-08-08T11:47:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25123 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:47:31 dist-test-slave-39l8 krb5kdc[3871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:47:31 dist-test-slave-39l8 krb5kdc[3871](info): set up 2 sockets
Aug 08 11:47:31 dist-test-slave-39l8 krb5kdc[3871](info): commencing operation
krb5kdc: starting...
W20260808 11:47:33.126888 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.006s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:47:33 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:47:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:47:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:47:33.421669 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:33657
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:33.758800 3895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:33.759086 3895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:33.759254 3895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:33.766373 3895 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 11:47:33.766471 3895 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 11:47:33.766543 3895 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 11:47:33.766619 3895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:33.766726 3895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:33.766840 3895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:33.766902 3895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:33.767004 3895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:33.776870 3895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:33.778242 3895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:33.779599 3895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:33.791778 3903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:33.791726 3901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:33.791827 3900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:33.792445 3895 server_base.cc:1034] running on GCE node
I20260808 11:47:33.793231 3895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:33.794628 3895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:33.795897 3895 hybrid_clock.cc:648] HybridClock initialized: now 1786189653795845 us; error 33 us; skew 500 ppm
I20260808 11:47:33.796221 3895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:33 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189653, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:33.804028 3895 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:47:33.805452 3895 webserver.cc:460] Webserver started at http://127.13.158.126:42973/ using document root <none> and password file <none>
I20260808 11:47:33.806089 3895 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:33.806182 3895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:33.806437 3895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:33.809350 3895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf"
format_stamp: "Formatted at 2026-08-08 11:47:33 on dist-test-slave-39l8"
server_key: "eb1655c8801d46ef4a1de3222e245e9b"
server_key_iv: "15dd8a25b37a0f0605bf5757a4e1faf2"
server_key_version: "encryptionkey@0"
I20260808 11:47:33.810122 3895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf"
format_stamp: "Formatted at 2026-08-08 11:47:33 on dist-test-slave-39l8"
server_key: "eb1655c8801d46ef4a1de3222e245e9b"
server_key_iv: "15dd8a25b37a0f0605bf5757a4e1faf2"
server_key_version: "encryptionkey@0"
I20260808 11:47:33.815192 3895 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 11:47:33.818502 3913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:33.819787 3895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:33.819947 3895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf"
format_stamp: "Formatted at 2026-08-08 11:47:33 on dist-test-slave-39l8"
server_key: "eb1655c8801d46ef4a1de3222e245e9b"
server_key_iv: "15dd8a25b37a0f0605bf5757a4e1faf2"
server_key_version: "encryptionkey@0"
I20260808 11:47:33.820164 3895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:33.836153 3895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:33.840058 3895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:33.840346 3895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:33.859030 3895 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:33657
I20260808 11:47:33.859054 3988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:33657 every 8 connection(s)
I20260808 11:47:33.860358 3895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:47:33.860911 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 3895
I20260808 11:47:33.861155 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:47:33.862030 13945 external_mini_cluster.cc:1428] Setting key c13c7fe2aa376cc56037c908040e74b1
I20260808 11:47:33.864715 3989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:47:33 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:33.877811 3989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Bootstrap starting.
I20260808 11:47:33.881417 3989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:33.882697 3989 log.cc:824] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:33.885921 3989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: No bootstrap required, opened a new log
I20260808 11:47:33.886116 3992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:33.864103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33862 (local address 127.13.158.126:33657)
0808 11:47:33.864898 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:33.864913 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:33.864961 (+ 48us) server_negotiation.cc:407] Connection header received
0808 11:47:33.867327 (+ 2366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:33.867354 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:33.867828 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:33.868260 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:33.869603 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:33.871697 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:33.873028 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:33.874027 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:33.874223 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:33.878573 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:33.878581 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:33.878593 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:33.878632 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:33.881794 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:33.882587 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:33.882593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:33.882598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:33.882815 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:33.883279 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:33.883286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:33.883292 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:33.884051 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:33.884303 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:33.884734 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 11:47:33.885097 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":329,"threads_started":1}
I20260808 11:47:33.893949 3989 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:47:33.894276 3989 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:33.894373 3989 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b148f85e5240e19bc4a4e21ae3ebcf, State: Initialized, Role: FOLLOWER
I20260808 11:47:33.895031 3989 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:47:33.895305 3989 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:47:33.895498 3989 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:47:33.895682 3989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:33.897837 3989 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:47:33.898278 3989 leader_election.cc:302] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b148f85e5240e19bc4a4e21ae3ebcf; no voters:
I20260808 11:47:33.898849 3989 leader_election.cc:288] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:47:33.898957 3995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:47:33.900297 3995 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 1 LEADER]: Becoming Leader. State: Replica: c3b148f85e5240e19bc4a4e21ae3ebcf, State: Running, Role: LEADER
I20260808 11:47:33.900780 3995 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:47:33.900950 3989 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:47:33.903836 3999 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b148f85e5240e19bc4a4e21ae3ebcf. Latest consensus state: current_term: 1 leader_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } } }
I20260808 11:47:33.904246 3999 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:33.904814 3997 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } } }
I20260808 11:47:33.905501 3997 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:33.905774 4008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:47:33.911459 4008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:47:33.919432 4008 catalog_manager.cc:1347] Generated new cluster ID: 32a6626ddf0d4af2b969288302a8648c
I20260808 11:47:33.919548 4008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:47:33.958441 4008 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:47:33.959851 4008 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:47:33.970422 4008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Generated new TSK 0
I20260808 11:47:33.971262 4008 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:47:34.052265 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33657
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:34.395454 4029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:34.395717 4029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:34.395900 4029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:34.403367 4029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:34.403494 4029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:34.403690 4029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:47:34.413781 4029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:34.415164 4029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:34.416486 4029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:34.429539 4037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:34.429566 4039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:34.429849 4042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:34.431159 4029 server_base.cc:1034] running on GCE node
I20260808 11:47:34.431905 4029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:34.433058 4029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:34.434285 4029 hybrid_clock.cc:648] HybridClock initialized: now 1786189654434245 us; error 30 us; skew 500 ppm
I20260808 11:47:34.434597 4029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:34 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:34.437649 4029 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:47:34.439198 4029 webserver.cc:460] Webserver started at http://127.13.158.65:38541/ using document root <none> and password file <none>
I20260808 11:47:34.439893 4029 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:34.440068 4029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:34.440398 4029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:34.443404 4029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "204657dc50e393c9ddf8d8ca8dc11222"
server_key_iv: "d3ecc3b5b102ede9130eb0fb38c402ce"
server_key_version: "encryptionkey@0"
I20260808 11:47:34.444154 4029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "204657dc50e393c9ddf8d8ca8dc11222"
server_key_iv: "d3ecc3b5b102ede9130eb0fb38c402ce"
server_key_version: "encryptionkey@0"
I20260808 11:47:34.449180 4029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:47:34.452548 4051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:34.453732 4029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:47:34.453913 4029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "204657dc50e393c9ddf8d8ca8dc11222"
server_key_iv: "d3ecc3b5b102ede9130eb0fb38c402ce"
server_key_version: "encryptionkey@0"
I20260808 11:47:34.454105 4029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:34.466856 4029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:34.471159 4029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:34.471464 4029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:34.472474 4029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:34.473842 4029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:34.473917 4029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:34.474001 4029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:34.474057 4029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:34.507663 4029 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:42487
I20260808 11:47:34.507712 4217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:42487 every 8 connection(s)
I20260808 11:47:34.508956 4029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:47:34.511495 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 4029
I20260808 11:47:34.511711 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:47:34.512655 13945 external_mini_cluster.cc:1428] Setting key 0a6c7df67ac9b9e3f7d2f2e0a7eb3808
Aug 08 11:47:34 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:34.532686 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:34.511577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:36937 (local address 127.13.158.126:33657)
0808 11:47:34.512136 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:34.512144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:34.513249 (+ 1105us) server_negotiation.cc:407] Connection header received
0808 11:47:34.515726 (+ 2477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:34.515730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:34.515793 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:34.515899 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:34.518001 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:34.518766 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:34.521147 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:34.521582 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:34.521679 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:34.526063 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:34.526068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:34.526071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:34.526123 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:34.529388 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:34.530312 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:34.530315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:34.530317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:34.530430 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:34.530947 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:34.530950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:34.530952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:34.531206 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:34.531322 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:34.532275 (+ 953us) server_negotiation.cc:299] Negotiation successful
0808 11:47:34.532441 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":250,"threads_started":1}
I20260808 11:47:34.534219 4223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:34.512156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:36937)
0808 11:47:34.513065 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:34.513115 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:34.515409 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:34.516155 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:34.516168 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:34.516962 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:34.517827 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:34.517854 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:34.518916 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:34.518923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:34.521002 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:34.521020 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:34.521827 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:34.521836 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:34.522171 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:34.523023 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:34.523058 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:34.525831 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:34.529543 (+ 3712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:34.529555 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:34.529574 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:34.530186 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:34.530531 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:34.530538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:34.530544 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:34.530832 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:34.531336 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:34.531346 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:34.531804 (+ 458us) client_negotiation.cc:769] Sending connection context
0808 11:47:34.532174 (+ 370us) client_negotiation.cc:240] Negotiation successful
0808 11:47:34.532746 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":300,"threads_started":1}
I20260808 11:47:34.536029 4218 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33657
I20260808 11:47:34.536372 4218 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:34.537375 4218 heartbeater.cc:511] Master 127.13.158.126:33657 requested a full tablet report, sending...
I20260808 11:47:34.539760 3933 ts_manager.cc:194] Registered new tserver with Master: f43ff67b9a9a4f56ab9f8fce8e7b4c65 (127.13.158.65:42487)
I20260808 11:47:34.541956 3933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:36937
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:47:34.581954 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33657
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:34.920738 4226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:34.920996 4226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:34.921236 4226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:34.928414 4226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:34.928563 4226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:34.928755 4226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:47:34.938175 4226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:34.939477 4226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:34.940868 4226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:34.954839 4232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:34.954887 4235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:34.954932 4233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:34.955726 4226 server_base.cc:1034] running on GCE node
I20260808 11:47:34.956379 4226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:34.957304 4226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:34.958554 4226 hybrid_clock.cc:648] HybridClock initialized: now 1786189654958496 us; error 60 us; skew 500 ppm
I20260808 11:47:34.958920 4226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:34 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:34.962038 4226 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:47:34.963591 4226 webserver.cc:460] Webserver started at http://127.13.158.66:33597/ using document root <none> and password file <none>
I20260808 11:47:34.964303 4226 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:34.964468 4226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:34.964789 4226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:34.967747 4226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "2b36bdbb96cd4018a168b925706c178f"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "bea281189b602658c72321907367bfad"
server_key_iv: "ea7547bb8984386679ff4d0bd8cc6ef6"
server_key_version: "encryptionkey@0"
I20260808 11:47:34.968537 4226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "2b36bdbb96cd4018a168b925706c178f"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "bea281189b602658c72321907367bfad"
server_key_iv: "ea7547bb8984386679ff4d0bd8cc6ef6"
server_key_version: "encryptionkey@0"
I20260808 11:47:34.973742 4226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:47:34.977309 4244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:34.978665 4226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:47:34.978935 4226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "2b36bdbb96cd4018a168b925706c178f"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "bea281189b602658c72321907367bfad"
server_key_iv: "ea7547bb8984386679ff4d0bd8cc6ef6"
server_key_version: "encryptionkey@0"
I20260808 11:47:34.979158 4226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:34.993045 4226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:34.997565 4226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:34.997862 4226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:34.998890 4226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:35.000258 4226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:35.000360 4226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:35.000511 4226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:35.000587 4226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:35.035429 4226 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:46095
I20260808 11:47:35.035467 4425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:46095 every 8 connection(s)
I20260808 11:47:35.036751 4226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:47:35.042644 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 4226
I20260808 11:47:35.042903 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:47:35.043404 13945 external_mini_cluster.cc:1428] Setting key 9488ab32b14a0c72ed090bba594d9587
Aug 08 11:47:35 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189654, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:35.059072 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.039317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:43011 (local address 127.13.158.126:33657)
0808 11:47:35.039779 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.039783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.040647 (+ 864us) server_negotiation.cc:407] Connection header received
0808 11:47:35.042522 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.042528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.042624 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.042762 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:35.044519 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.045477 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.047139 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.047921 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.048072 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.051756 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:35.051766 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:35.051773 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:35.051821 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:35.055585 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.056361 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.056368 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.056374 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.056555 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.056984 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.056991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.056997 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.057403 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:35.057560 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.058401 (+ 841us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.058670 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":255,"threads_started":1}
I20260808 11:47:35.060520 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.039785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:43011)
0808 11:47:35.040482 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:35.040530 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:35.042229 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:35.043058 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:35.043067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:35.043620 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:35.044340 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.044362 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.045630 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.045634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.046984 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.046997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.048128 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.048134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.048322 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.049033 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:35.049056 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:35.051514 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:35.055748 (+ 4234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:35.055755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:35.055770 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:35.056209 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:35.056665 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:35.056669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:35.056671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:35.056868 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:35.057532 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:35.057539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:35.057930 (+ 391us) client_negotiation.cc:769] Sending connection context
0808 11:47:35.058240 (+ 310us) client_negotiation.cc:240] Negotiation successful
0808 11:47:35.058876 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":276,"threads_started":1}
I20260808 11:47:35.062055 4426 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33657
I20260808 11:47:35.062388 4426 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:35.063246 4426 heartbeater.cc:511] Master 127.13.158.126:33657 requested a full tablet report, sending...
I20260808 11:47:35.064839 3933 ts_manager.cc:194] Registered new tserver with Master: 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095)
I20260808 11:47:35.066147 3933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:43011
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:47:35.110065 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33657
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:35.451052 4434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:35.451363 4434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:35.451530 4434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:35.458927 4434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:35.459064 4434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:35.459236 4434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:47:35.468838 4434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:35.470127 4434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:35.471531 4434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:35.484781 4444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:35.484899 4440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:35.484874 4441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:35.485694 4434 server_base.cc:1034] running on GCE node
I20260808 11:47:35.486291 4434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:35.487262 4434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:35.488484 4434 hybrid_clock.cc:648] HybridClock initialized: now 1786189655488439 us; error 57 us; skew 500 ppm
I20260808 11:47:35.488795 4434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:35 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189655, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:35.491928 4434 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:47:35.493332 4434 webserver.cc:460] Webserver started at http://127.13.158.67:37945/ using document root <none> and password file <none>
I20260808 11:47:35.494029 4434 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:35.494185 4434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:35.494508 4434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:47:35.497442 4434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "6a07ea4184fa421f976f80be31a78f53"
format_stamp: "Formatted at 2026-08-08 11:47:35 on dist-test-slave-39l8"
server_key: "c3871ce5ee5f44a4fe89a8ccf190d89a"
server_key_iv: "d13cfad7f8c8853064d10a0148710410"
server_key_version: "encryptionkey@0"
I20260808 11:47:35.498193 4434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "6a07ea4184fa421f976f80be31a78f53"
format_stamp: "Formatted at 2026-08-08 11:47:35 on dist-test-slave-39l8"
server_key: "c3871ce5ee5f44a4fe89a8ccf190d89a"
server_key_iv: "d13cfad7f8c8853064d10a0148710410"
server_key_version: "encryptionkey@0"
I20260808 11:47:35.503506 4434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 11:47:35.506919 4453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:35.508545 4434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:47:35.508692 4434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "6a07ea4184fa421f976f80be31a78f53"
format_stamp: "Formatted at 2026-08-08 11:47:35 on dist-test-slave-39l8"
server_key: "c3871ce5ee5f44a4fe89a8ccf190d89a"
server_key_iv: "d13cfad7f8c8853064d10a0148710410"
server_key_version: "encryptionkey@0"
I20260808 11:47:35.508890 4434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:35.525362 4434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:35.529229 4434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:35.529470 4434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:35.530370 4434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:35.531723 4434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:35.531797 4434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:35.531883 4434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:35.531927 4434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:35.545800 4218 heartbeater.cc:503] Master 127.13.158.126:33657 was elected leader, sending a full tablet report...
I20260808 11:47:35.566154 4434 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:37611
I20260808 11:47:35.566186 4624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:37611 every 8 connection(s)
I20260808 11:47:35.567564 4434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:47:35.569438 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 4434
I20260808 11:47:35.569651 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:47:35.570267 13945 external_mini_cluster.cc:1428] Setting key e9ad36cfc4756e8ed4a382e6dbbaf2b0
Aug 08 11:47:35 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189655, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:35.588691 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.570303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:53319 (local address 127.13.158.126:33657)
0808 11:47:35.570849 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.570870 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.572109 (+ 1239us) server_negotiation.cc:407] Connection header received
0808 11:47:35.574573 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.574579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.574665 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.574819 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:35.576300 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.577069 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.578596 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.579079 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.579226 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.582617 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:35.582630 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:35.582634 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:35.582667 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:35.585824 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.586637 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.586641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.586643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.586833 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.587220 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.587224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.587226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.587479 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:35.587591 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.588233 (+ 642us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.588397 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":255,"threads_started":1}
I20260808 11:47:35.589794 4629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.571001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:53319)
0808 11:47:35.571958 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:35.572009 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:35.574296 (+ 2287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:35.575064 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:35.575071 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:35.575570 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:35.576131 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.576148 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.577226 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.577231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.578453 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.578465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.579311 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.579319 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.579569 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.580250 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:35.580272 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:35.582424 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:35.585980 (+ 3556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:35.585992 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:35.586009 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:35.586530 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:35.586938 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:35.586942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:35.586944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:35.587126 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:35.587592 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:35.587598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:35.587932 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 11:47:35.588154 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 11:47:35.588532 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":368,"threads_started":1}
I20260808 11:47:35.591168 4625 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33657
I20260808 11:47:35.591461 4625 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:35.592216 4625 heartbeater.cc:511] Master 127.13.158.126:33657 requested a full tablet report, sending...
I20260808 11:47:35.593583 3933 ts_manager.cc:194] Registered new tserver with Master: 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
I20260808 11:47:35.594399 3933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:53319
I20260808 11:47:35.598387 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:47:35.615945 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.603119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.13.158.126:33657)
0808 11:47:35.603276 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.603280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.603790 (+ 510us) server_negotiation.cc:407] Connection header received
0808 11:47:35.603985 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.603989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.604061 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.604141 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:35.605735 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.606454 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.607976 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.608389 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.608481 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.610924 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:35.610929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:35.610932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:35.610962 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:35.613377 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.614044 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.614048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.614050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.614184 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.614585 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.614588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.614590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.614902 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:35.615097 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.615437 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.615606 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:47:35.620144 3933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:47:35.627447 3933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:47:35.650060 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.641583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42430 (local address 127.13.158.67:37611)
0808 11:47:35.642438 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.642445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.642477 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:47:35.642576 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.642583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.642842 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.643029 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:35.643644 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.644862 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.647554 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.648492 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.648625 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.648882 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.649520 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.649680 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":1239,"threads_started":1}
I20260808 11:47:35.650215 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.639549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32818)
0808 11:47:35.641364 (+ 1815us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:35.641385 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:35.641622 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:35.642501 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:35.642505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:35.642545 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:35.642944 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.642956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.644755 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.644761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.647035 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.647048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.649264 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.649270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.649544 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.649583 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 11:47:35.649676 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 11:47:35.649809 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1674,"mutex_wait_us":157,"thread_start_us":1537,"threads_started":1}
I20260808 11:47:35.650290 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.639517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.13.158.66:46095)
0808 11:47:35.640156 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.640168 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.640906 (+ 738us) server_negotiation.cc:407] Connection header received
0808 11:47:35.641376 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.641384 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.641622 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.641796 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:35.642655 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.644321 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.647331 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.648711 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.648849 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.649043 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.649775 (+ 732us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.649949 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":307,"threads_started":1}
I20260808 11:47:35.650403 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.640082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.13.158.65:42487)
0808 11:47:35.640898 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.640910 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.641499 (+ 589us) server_negotiation.cc:407] Connection header received
0808 11:47:35.641834 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.641842 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.642047 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.642215 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:35.643152 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.644623 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.647190 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.648711 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.648848 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.649067 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.649938 (+ 871us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.650101 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":493,"threads_started":1}
I20260808 11:47:35.650728 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.640998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37611 (local address 127.0.0.1:42430)
0808 11:47:35.641644 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:35.641661 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:35.641758 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:35.643063 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:35.643069 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:35.643101 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:35.643478 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.643487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.645103 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.645114 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.647405 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.647416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.649020 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.649027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.649243 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.649312 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 11:47:35.649459 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 11:47:35.649675 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":6,"thread_start_us":902,"threads_started":1}
I20260808 11:47:35.651182 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.639198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:46095 (local address 127.0.0.1:43836)
0808 11:47:35.640767 (+ 1569us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:35.640807 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:35.641058 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:35.641951 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:35.641958 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:35.641983 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:35.642453 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.642467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.644465 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.644471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.647173 (+ 2702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.647188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.649227 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.649236 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.649548 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.649596 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 11:47:35.650535 (+ 939us) client_negotiation.cc:240] Negotiation successful
0808 11:47:35.650690 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1330,"mutex_wait_us":130,"thread_start_us":1046,"threads_started":1}
I20260808 11:47:35.654493 4323 tablet_service.cc:1467] Processing CreateTablet for tablet 2706607b5ab44cf4b48faf68b336dc9c (DEFAULT_TABLE table=test-table [id=98b1423f301541e28a1bbaaeea2dfe2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:35.654623 4120 tablet_service.cc:1467] Processing CreateTablet for tablet 2706607b5ab44cf4b48faf68b336dc9c (DEFAULT_TABLE table=test-table [id=98b1423f301541e28a1bbaaeea2dfe2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:35.654624 4529 tablet_service.cc:1467] Processing CreateTablet for tablet 2706607b5ab44cf4b48faf68b336dc9c (DEFAULT_TABLE table=test-table [id=98b1423f301541e28a1bbaaeea2dfe2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:35.655833 4323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2706607b5ab44cf4b48faf68b336dc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:35.656045 4120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2706607b5ab44cf4b48faf68b336dc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:35.656045 4529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2706607b5ab44cf4b48faf68b336dc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:35.667485 4653 tablet_bootstrap.cc:492] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Bootstrap starting.
I20260808 11:47:35.671572 4653 tablet_bootstrap.cc:654] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:35.671891 4655 tablet_bootstrap.cc:492] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Bootstrap starting.
I20260808 11:47:35.673031 4653 log.cc:824] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:35.670622 4654 tablet_bootstrap.cc:492] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: Bootstrap starting.
I20260808 11:47:35.675601 4655 tablet_bootstrap.cc:654] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:35.675839 4653 tablet_bootstrap.cc:492] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: No bootstrap required, opened a new log
I20260808 11:47:35.676103 4653 ts_tablet_manager.cc:1397] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 11:47:35.676815 4655 log.cc:824] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:35.677299 4654 tablet_bootstrap.cc:654] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:35.678512 4654 log.cc:824] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:35.680119 4655 tablet_bootstrap.cc:492] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: No bootstrap required, opened a new log
I20260808 11:47:35.680503 4655 ts_tablet_manager.cc:1397] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 11:47:35.684995 4653 raft_consensus.cc:348] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.685355 4653 raft_consensus.cc:374] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:35.685482 4653 raft_consensus.cc:729] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b36bdbb96cd4018a168b925706c178f, State: Initialized, Role: FOLLOWER
I20260808 11:47:35.685642 4654 tablet_bootstrap.cc:492] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: No bootstrap required, opened a new log
I20260808 11:47:35.685906 4654 ts_tablet_manager.cc:1397] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20260808 11:47:35.686157 4653 consensus_queue.cc:260] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.687337 4426 heartbeater.cc:503] Master 127.13.158.126:33657 was elected leader, sending a full tablet report...
I20260808 11:47:35.687799 4653 ts_tablet_manager.cc:1428] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260808 11:47:35.691332 4655 raft_consensus.cc:348] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.691748 4655 raft_consensus.cc:374] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:35.691857 4655 raft_consensus.cc:729] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43ff67b9a9a4f56ab9f8fce8e7b4c65, State: Initialized, Role: FOLLOWER
I20260808 11:47:35.692446 4655 consensus_queue.cc:260] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.694123 4655 ts_tablet_manager.cc:1428] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 11:47:35.696794 4654 raft_consensus.cc:348] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.697212 4654 raft_consensus.cc:374] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:35.697332 4654 raft_consensus.cc:729] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a07ea4184fa421f976f80be31a78f53, State: Initialized, Role: FOLLOWER
I20260808 11:47:35.697827 4654 consensus_queue.cc:260] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.698846 4625 heartbeater.cc:503] Master 127.13.158.126:33657 was elected leader, sending a full tablet report...
I20260808 11:47:35.699450 4654 ts_tablet_manager.cc:1428] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 11:47:35.735745 4660 raft_consensus.cc:482] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:47:35.736008 4660 raft_consensus.cc:504] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.737681 4660 leader_election.cc:288] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
I20260808 11:47:35.744589 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.738282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:45133 (local address 127.13.158.66:46095)
0808 11:47:35.738559 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.738567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.739036 (+ 469us) server_negotiation.cc:407] Connection header received
0808 11:47:35.739338 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.739345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.739434 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.739509 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:35.740344 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.741107 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.742893 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.743661 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.743836 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.743915 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.744192 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.744326 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 11:47:35.744657 4662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.738090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:46095 (local address 127.13.158.65:45133)
0808 11:47:35.738927 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:35.738948 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:35.739164 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:35.739755 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:35.739760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:35.739785 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:35.740204 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.740216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.741236 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.741243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.742679 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.742692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.743875 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.743881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.744036 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.744059 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:47:35.744166 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 11:47:35.744290 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":435,"threads_started":1}
I20260808 11:47:35.745098 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.739150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:54827 (local address 127.13.158.67:37611)
0808 11:47:35.739395 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.739403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.739942 (+ 539us) server_negotiation.cc:407] Connection header received
0808 11:47:35.740222 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.740226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.740283 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.740383 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:35.740982 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.741686 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.743403 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.744164 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.744350 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.744423 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.744731 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.744857 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 11:47:35.745137 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.739022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37611 (local address 127.13.158.65:54827)
0808 11:47:35.739828 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:35.739851 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:35.740035 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:35.740403 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:35.740407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:35.740431 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:35.740757 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.740766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.741839 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.741844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.743248 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:35.743261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.744387 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.744394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:35.744547 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.744570 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:47:35.744699 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 11:47:35.744818 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":479,"threads_started":1}
I20260808 11:47:35.745884 4356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2706607b5ab44cf4b48faf68b336dc9c" candidate_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b36bdbb96cd4018a168b925706c178f" is_pre_election: true
I20260808 11:47:35.746299 4560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2706607b5ab44cf4b48faf68b336dc9c" candidate_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a07ea4184fa421f976f80be31a78f53" is_pre_election: true
I20260808 11:47:35.746483 4356 raft_consensus.cc:2454] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f43ff67b9a9a4f56ab9f8fce8e7b4c65 in term 0.
I20260808 11:47:35.746882 4560 raft_consensus.cc:2454] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f43ff67b9a9a4f56ab9f8fce8e7b4c65 in term 0.
I20260808 11:47:35.747244 4057 leader_election.cc:302] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b36bdbb96cd4018a168b925706c178f, f43ff67b9a9a4f56ab9f8fce8e7b4c65; no voters:
I20260808 11:47:35.747634 4660 raft_consensus.cc:2781] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:47:35.747802 4660 raft_consensus.cc:482] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:47:35.747934 4660 raft_consensus.cc:3037] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:35.750118 4660 raft_consensus.cc:504] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.750798 4660 leader_election.cc:288] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 1 election: Requested vote from peers 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
I20260808 11:47:35.751163 4356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2706607b5ab44cf4b48faf68b336dc9c" candidate_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b36bdbb96cd4018a168b925706c178f"
I20260808 11:47:35.751240 4560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2706607b5ab44cf4b48faf68b336dc9c" candidate_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a07ea4184fa421f976f80be31a78f53"
I20260808 11:47:35.751339 4356 raft_consensus.cc:3037] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:35.751482 4560 raft_consensus.cc:3037] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:35.753648 4356 raft_consensus.cc:2454] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f43ff67b9a9a4f56ab9f8fce8e7b4c65 in term 1.
I20260808 11:47:35.753676 4560 raft_consensus.cc:2454] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f43ff67b9a9a4f56ab9f8fce8e7b4c65 in term 1.
I20260808 11:47:35.754038 4057 leader_election.cc:302] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b36bdbb96cd4018a168b925706c178f, f43ff67b9a9a4f56ab9f8fce8e7b4c65; no voters:
I20260808 11:47:35.754302 4660 raft_consensus.cc:2781] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:47:35.754694 4660 raft_consensus.cc:686] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 LEADER]: Becoming Leader. State: Replica: f43ff67b9a9a4f56ab9f8fce8e7b4c65, State: Running, Role: LEADER
I20260808 11:47:35.755476 4660 consensus_queue.cc:237] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:35.759994 3932 catalog_manager.cc:5495] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to f43ff67b9a9a4f56ab9f8fce8e7b4c65 (127.13.158.65). New cstate: current_term: 1 leader_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } health_report { overall_health: UNKNOWN } } }
W20260808 11:47:35.761401 4220 tablet.cc:2369] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:35.776044 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.770469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.13.158.65:42487)
0808 11:47:35.770663 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.770670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.770771 (+ 101us) server_negotiation.cc:407] Connection header received
0808 11:47:35.771022 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.771026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.771099 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.771288 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:47:35.771710 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.772438 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.773798 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.774205 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.774349 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.774941 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:47:35.775450 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:47:35.775619 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.775677 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.775763 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:47:35.784053 4560 raft_consensus.cc:1261] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 1 FOLLOWER]: Refusing update from remote peer f43ff67b9a9a4f56ab9f8fce8e7b4c65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:35.784142 4356 raft_consensus.cc:1261] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 1 FOLLOWER]: Refusing update from remote peer f43ff67b9a9a4f56ab9f8fce8e7b4c65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:35.785125 4664 consensus_queue.cc:1035] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:47:35.785486 4660 consensus_queue.cc:1035] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:47:35.788733 4427 tablet.cc:2369] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:35.797441 4668 mvcc.cc:205] Tried to move back new op lower bound from 7316232830080860160 to 7316232829978030080. Current Snapshot: MvccSnapshot[applied={T|T < 7316232830080860160}]
I20260808 11:47:35.800845 4670 mvcc.cc:205] Tried to move back new op lower bound from 7316232830080860160 to 7316232829978030080. Current Snapshot: MvccSnapshot[applied={T|T < 7316232830080860160}]
I20260808 11:47:35.813275 3931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:47:35.813604 3931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:47:35.817473 3931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Sending DeleteTablet for 3 replicas of tablet 2706607b5ab44cf4b48faf68b336dc9c
I20260808 11:47:35.818759 4120 tablet_service.cc:1514] Processing DeleteTablet for tablet 2706607b5ab44cf4b48faf68b336dc9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:35 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:32818
I20260808 11:47:35.818929 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 4029
I20260808 11:47:35.818887 4529 tablet_service.cc:1514] Processing DeleteTablet for tablet 2706607b5ab44cf4b48faf68b336dc9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:35 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:42430
I20260808 11:47:35.819556 4323 tablet_service.cc:1514] Processing DeleteTablet for tablet 2706607b5ab44cf4b48faf68b336dc9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:35 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:43836
I20260808 11:47:35.829921 4686 tablet_replica.cc:330] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: stopping tablet replica
I20260808 11:47:35.831233 4686 raft_consensus.cc:2229] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:47:35.832590 4686 raft_consensus.cc:2258] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:35.832792 4682 tablet_replica.cc:330] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: stopping tablet replica
I20260808 11:47:35.833184 4682 raft_consensus.cc:2229] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:47:35.833851 4682 raft_consensus.cc:2258] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:35.835537 4686 ts_tablet_manager.cc:1905] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:35.838001 4683 tablet_replica.cc:330] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: stopping tablet replica
I20260808 11:47:35.838387 4683 raft_consensus.cc:2229] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:47:35.840680 4682 ts_tablet_manager.cc:1905] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:35.843303 4686 ts_tablet_manager.cc:1918] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:35.843478 4686 log.cc:1196] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/2706607b5ab44cf4b48faf68b336dc9c
I20260808 11:47:35.844080 4686 ts_tablet_manager.cc:1939] T 2706607b5ab44cf4b48faf68b336dc9c P 2b36bdbb96cd4018a168b925706c178f: Deleting consensus metadata
Aug 08 11:47:35 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:47:35.846105 3916 catalog_manager.cc:4841] TS 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095): tablet 2706607b5ab44cf4b48faf68b336dc9c (table test-table [id=98b1423f301541e28a1bbaaeea2dfe2e]) successfully deleted
I20260808 11:47:35.846902 4683 raft_consensus.cc:2258] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:35.849056 4683 ts_tablet_manager.cc:1905] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:35.852464 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.820030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32826 (local address 127.13.158.65:42487)
0808 11:47:35.820242 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.820248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.829436 (+ 9188us) server_negotiation.cc:407] Connection header received
0808 11:47:35.829912 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.829918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.830023 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.830100 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:35.834271 (+ 4171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.835529 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.838875 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.839587 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.839706 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.845767 (+ 6061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:35.845785 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:35.845801 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:35.845865 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:35.849411 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.850170 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.850176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.850178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.850349 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.850807 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.850810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.850812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.851130 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:35.851454 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.851814 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.852077 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:47:35.853149 4211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:35.856093 4683 ts_tablet_manager.cc:1918] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:35.856261 4683 log.cc:1196] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/2706607b5ab44cf4b48faf68b336dc9c
I20260808 11:47:35.856844 4683 ts_tablet_manager.cc:1939] T 2706607b5ab44cf4b48faf68b336dc9c P 6a07ea4184fa421f976f80be31a78f53: Deleting consensus metadata
I20260808 11:47:35.859803 3916 catalog_manager.cc:4841] TS 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611): tablet 2706607b5ab44cf4b48faf68b336dc9c (table test-table [id=98b1423f301541e28a1bbaaeea2dfe2e]) successfully deleted
I20260808 11:47:35.950716 4682 ts_tablet_manager.cc:1918] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:35.951141 4682 log.cc:1196] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/2706607b5ab44cf4b48faf68b336dc9c
I20260808 11:47:35.951229 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 4029
I20260808 11:47:35.975030 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 4226
W20260808 11:47:35.975039 3917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.158.65:42487 (error 108)
I20260808 11:47:35.976977 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.976551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32840)
0808 11:47:35.976743 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:35.976839 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 11:47:35.977186 3917 catalog_manager.cc:4584] TS f43ff67b9a9a4f56ab9f8fce8e7b4c65 (127.13.158.65:42487): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2706607b5ab44cf4b48faf68b336dc9c: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Aug 08 11:47:35 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:47:35.991039 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:35.975930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.13.158.66:46095)
0808 11:47:35.976118 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:35.976124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:35.976193 (+ 69us) server_negotiation.cc:407] Connection header received
0808 11:47:35.976359 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:35.976365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:35.976451 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:35.976540 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:35.978073 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.979097 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.980334 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:35.980709 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:35.980800 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:35.985516 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:35.985529 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:35.985545 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:35.985585 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:35.988477 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.989108 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.989115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.989117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.989239 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:35.989547 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:35.989550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:35.989552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:35.990023 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:35.990339 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:35.990499 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 11:47:35.990662 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:47:35.991689 4415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:36.030807 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:36.030289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32842)
0808 11:47:36.030497 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:36.030590 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 11:47:36.031831 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:36.031478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32848)
0808 11:47:36.031627 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:36.031706 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 11:47:36.083913 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 4226
I20260808 11:47:36.092221 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:36.091735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32860)
0808 11:47:36.091964 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:36.092061 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 11:47:36.093230 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:36.092869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32876)
0808 11:47:36.093023 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:36.093096 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 11:47:36.109174 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 4434
Aug 08 11:47:36 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:47:36.125324 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:36.110012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.13.158.67:37611)
0808 11:47:36.110182 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:36.110187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:36.110208 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:47:36.110363 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:36.110367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:36.110441 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:36.110525 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:36.111970 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:36.112740 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:36.114038 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:36.114495 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:36.114586 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:36.119302 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:36.119318 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:36.119334 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:36.119367 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:36.122202 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:36.123240 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:36.123248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:36.123250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:36.123439 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:36.123897 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:36.123900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:36.123902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:36.124250 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:36.124529 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:36.124807 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 11:47:36.125006 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:47:36.125983 4616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:36.173951 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:36.173481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32884)
0808 11:47:36.173702 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:36.173810 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 11:47:36.175053 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:36.174616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32892)
0808 11:47:36.174867 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:36.174937 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260808 11:47:36.215958 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 4434
I20260808 11:47:36.241859 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 3895
I20260808 11:47:36.243053 3984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:36.353158 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 3895
I20260808 11:47:36.353193 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:36.352672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:32894)
0808 11:47:36.352935 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:36.353031 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260808 11:47:56.383940 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:33657
--webserver_interface=127.13.158.126
--webserver_port=42973
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:47:56.722892 4696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:56.723228 4696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:56.723397 4696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:56.730142 4696 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 11:47:56.730270 4696 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 11:47:56.730388 4696 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 11:47:56.730485 4696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:47:56.730587 4696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:56.730687 4696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:56.730799 4696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:47:56.730882 4696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:47:56.740211 4696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=42973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:56.741482 4696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:56.742875 4696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:56.769112 4711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:56.769227 4705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:56.769198 4708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:56.772222 4696 server_base.cc:1034] running on GCE node
I20260808 11:47:56.773175 4696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:56.774909 4696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:56.776160 4696 hybrid_clock.cc:648] HybridClock initialized: now 1786189676776103 us; error 48 us; skew 500 ppm
I20260808 11:47:56.776531 4696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:56 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189676, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:56.785351 4696 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:47:56.787079 4696 webserver.cc:460] Webserver started at http://127.13.158.126:42973/ using document root <none> and password file <none>
I20260808 11:47:56.787863 4696 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:56.787973 4696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:56.794113 4696 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 11:47:56.797096 4725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:56.798478 4696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:47:56.798671 4696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf"
format_stamp: "Formatted at 2026-08-08 11:47:33 on dist-test-slave-39l8"
server_key: "eb1655c8801d46ef4a1de3222e245e9b"
server_key_iv: "15dd8a25b37a0f0605bf5757a4e1faf2"
server_key_version: "encryptionkey@0"
I20260808 11:47:56.799753 4696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:56.817544 4696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:56.821712 4696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:56.821967 4696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:56.841401 4696 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:33657
I20260808 11:47:56.841444 4794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:33657 every 8 connection(s)
I20260808 11:47:56.843050 4696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:47:56.845089 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 4696
I20260808 11:47:56.845876 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:42487
--local_ip_for_outbound_sockets=127.13.158.65
--tserver_master_addrs=127.13.158.126:33657
--webserver_port=38541
--webserver_interface=127.13.158.65
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:47:56.850653 4796 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:47:56.853955 4796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Bootstrap starting.
I20260808 11:47:56.873497 4796 log.cc:824] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:56.885761 4796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:47:56.886324 4796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Bootstrap complete.
I20260808 11:47:56.895649 4796 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:47:56.896438 4796 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b148f85e5240e19bc4a4e21ae3ebcf, State: Initialized, Role: FOLLOWER
I20260808 11:47:56.897161 4796 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:47:56.897406 4796 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:47:56.897555 4796 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:47:56.897756 4796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:47:56.901719 4796 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:47:56.902243 4796 leader_election.cc:302] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b148f85e5240e19bc4a4e21ae3ebcf; no voters:
I20260808 11:47:56.902933 4796 leader_election.cc:288] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 11:47:56.903128 4800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:47:56.905639 4796 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:47:56.907052 4800 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 2 LEADER]: Becoming Leader. State: Replica: c3b148f85e5240e19bc4a4e21ae3ebcf, State: Running, Role: LEADER
I20260808 11:47:56.907708 4800 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:47:56.910959 4802 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b148f85e5240e19bc4a4e21ae3ebcf. Latest consensus state: current_term: 2 leader_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } } }
I20260808 11:47:56.911037 4801 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } } }
I20260808 11:47:56.911402 4801 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:56.911403 4802 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: This master's current role is: LEADER
I20260808 11:47:56.919903 4807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:47:56.928176 4807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:47:56.929677 4807 catalog_manager.cc:1259] Loaded cluster ID: 32a6626ddf0d4af2b969288302a8648c
I20260808 11:47:56.929798 4807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:47:56.933926 4807 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:47:56.935407 4807 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Loaded TSK: 0
I20260808 11:47:56.950290 4826 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 11:47:56.972278 4807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Generated new TSK 1
I20260808 11:47:56.972708 4807 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 11:47:57.207733 4798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:57.208061 4798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:57.208230 4798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:57.215463 4798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:57.215619 4798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:57.215811 4798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:47:57.225662 4798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:42487
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=38541
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:57.226982 4798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:57.228343 4798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:57.243041 4835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:57.243114 4836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:57.243860 4838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:57.244439 4798 server_base.cc:1034] running on GCE node
I20260808 11:47:57.245093 4798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:57.246027 4798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:57.247253 4798 hybrid_clock.cc:648] HybridClock initialized: now 1786189677247188 us; error 58 us; skew 500 ppm
I20260808 11:47:57.247606 4798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:57 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189677, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:57.250958 4798 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:47:57.252460 4798 webserver.cc:460] Webserver started at http://127.13.158.65:38541/ using document root <none> and password file <none>
I20260808 11:47:57.253176 4798 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:57.253330 4798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:57.258911 4798 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 11:47:57.261766 4848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:57.263230 4798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:57.263424 4798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "204657dc50e393c9ddf8d8ca8dc11222"
server_key_iv: "d3ecc3b5b102ede9130eb0fb38c402ce"
server_key_version: "encryptionkey@0"
I20260808 11:47:57.264369 4798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:57.297633 4798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:57.301923 4798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:57.302199 4798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:57.303406 4798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:57.305260 4864 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260808 11:47:57.310667 4864 ts_tablet_manager.cc:1852] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 11:47:57.311707 4864 ts_tablet_manager.cc:1905] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:57.318822 4864 ts_tablet_manager.cc:1918] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:57.319049 4864 ts_tablet_manager.cc:1939] T 2706607b5ab44cf4b48faf68b336dc9c P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Deleting consensus metadata
I20260808 11:47:57.319391 4798 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 11:47:57.319517 4798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260808 11:47:57.319644 4798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:57.319759 4798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:57.352934 4798 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:42487
I20260808 11:47:57.353018 5015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:42487 every 8 connection(s)
I20260808 11:47:57.354230 4798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:47:57.356988 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 4798
I20260808 11:47:57.357833 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:46095
--local_ip_for_outbound_sockets=127.13.158.66
--tserver_master_addrs=127.13.158.126:33657
--webserver_port=33597
--webserver_interface=127.13.158.66
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:47:57 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189677, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:57.381651 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:57.357340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:58961)
0808 11:47:57.358384 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:57.358447 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:57.361278 (+ 2831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:57.362633 (+ 1355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:57.362645 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:57.363462 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:57.364257 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:57.364285 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:57.365797 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:57.365805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:57.367983 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:57.368001 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:57.368994 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:57.369003 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:57.369310 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:57.370032 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:57.370065 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:57.372634 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:57.377401 (+ 4767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:57.377408 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:57.377420 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:57.377828 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:57.378366 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:57.378369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:57.378371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:57.378529 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:57.379487 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:57.379492 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:57.379771 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 11:47:57.379981 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 11:47:57.380353 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":440,"threads_started":1}
I20260808 11:47:57.381922 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:57.357216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:58961 (local address 127.13.158.126:33657)
0808 11:47:57.358143 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:57.358183 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:57.358598 (+ 415us) server_negotiation.cc:407] Connection header received
0808 11:47:57.361804 (+ 3206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:57.361816 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:57.362130 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:57.362418 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:57.364463 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:57.365596 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:57.368126 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:57.368741 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:57.368939 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:57.372929 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:57.372941 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:57.372961 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:57.373019 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:57.377259 (+ 4240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:57.378007 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:57.378018 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:57.378027 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:57.378258 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:57.378657 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:57.378665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:57.378671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:57.379345 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:57.379526 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:57.380264 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 11:47:57.380708 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":271,"threads_started":1}
I20260808 11:47:57.384745 5016 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33657
I20260808 11:47:57.385067 5016 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:57.385963 5016 heartbeater.cc:511] Master 127.13.158.126:33657 requested a full tablet report, sending...
I20260808 11:47:57.388326 4748 ts_manager.cc:194] Registered new tserver with Master: f43ff67b9a9a4f56ab9f8fce8e7b4c65 (127.13.158.65:42487)
I20260808 11:47:57.389823 4748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:58961
W20260808 11:47:57.706470 5024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:57.706849 5024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:57.707031 5024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:57.714313 5024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:57.714422 5024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:57.714632 5024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:47:57.724413 5024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:46095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=33597
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:57.725680 5024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:57.727046 5024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:57.740232 5035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:57.740540 5033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:57.740813 5032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:57.741277 5024 server_base.cc:1034] running on GCE node
I20260808 11:47:57.741863 5024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:57.742784 5024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:57.744011 5024 hybrid_clock.cc:648] HybridClock initialized: now 1786189677743951 us; error 55 us; skew 500 ppm
I20260808 11:47:57.744349 5024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:57 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189677, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:57.747396 5024 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:47:57.748875 5024 webserver.cc:460] Webserver started at http://127.13.158.66:33597/ using document root <none> and password file <none>
I20260808 11:47:57.749567 5024 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:57.749732 5024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:57.755268 5024 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:47:57.758028 5049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:57.759456 5024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:57.759645 5024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "2b36bdbb96cd4018a168b925706c178f"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "bea281189b602658c72321907367bfad"
server_key_iv: "ea7547bb8984386679ff4d0bd8cc6ef6"
server_key_version: "encryptionkey@0"
I20260808 11:47:57.760639 5024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:57.775344 5024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:57.779413 5024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:57.779700 5024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:57.780687 5024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:57.782133 5024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:57.782271 5024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:57.782404 5024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:57.782496 5024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:57.816200 5024 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:46095
I20260808 11:47:57.816303 5217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:46095 every 8 connection(s)
I20260808 11:47:57.817665 5024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:47:57.823314 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 5024
I20260808 11:47:57.823916 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:37611
--local_ip_for_outbound_sockets=127.13.158.67
--tserver_master_addrs=127.13.158.126:33657
--webserver_port=37945
--webserver_interface=127.13.158.67
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:47:57 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189677, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:57.838356 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:57.820241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:35415 (local address 127.13.158.126:33657)
0808 11:47:57.820431 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:57.820436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:57.821592 (+ 1156us) server_negotiation.cc:407] Connection header received
0808 11:47:57.823506 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:57.823511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:57.823603 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:57.823716 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:57.825227 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:57.825934 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:57.827559 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:57.828003 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:57.828144 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:57.832183 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:57.832188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:57.832191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:57.832226 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:57.834829 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:57.835775 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:57.835783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:57.835788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:57.835953 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:57.836370 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:57.836376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:57.836380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:57.836796 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:57.836969 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:57.837630 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 11:47:57.837901 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:47:57.838963 5223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:57.820669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:35415)
0808 11:47:57.821414 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:57.821452 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:57.823235 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:57.823950 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:57.823958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:57.824513 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:57.825052 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:57.825070 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:57.826110 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:57.826114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:57.827400 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:57.827412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:57.828252 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:57.828259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:57.828471 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:57.829167 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:57.829189 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:57.831923 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:57.834971 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:57.834982 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:57.835000 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:57.835637 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:57.836063 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:57.836066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:57.836068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:57.836253 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:57.836932 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:57.836937 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:57.837244 (+ 307us) client_negotiation.cc:769] Sending connection context
0808 11:47:57.837466 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 11:47:57.837825 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":266,"threads_started":1}
I20260808 11:47:57.840613 5219 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33657
I20260808 11:47:57.840997 5219 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:57.841738 5219 heartbeater.cc:511] Master 127.13.158.126:33657 requested a full tablet report, sending...
I20260808 11:47:57.843209 4748 ts_manager.cc:194] Registered new tserver with Master: 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095)
I20260808 11:47:57.844024 4748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:35415
W20260808 11:47:58.184051 5226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:47:58.184496 5226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:47:58.184689 5226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:47:58.192266 5226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:47:58.192369 5226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:47:58.192487 5226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:47:58.202207 5226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:37611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=37945
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:47:58.203601 5226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:47:58.205008 5226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:47:58.218434 5234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:58.218467 5233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:47:58.218506 5238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:47:58.219244 5226 server_base.cc:1034] running on GCE node
I20260808 11:47:58.220031 5226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:47:58.221073 5226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:47:58.222316 5226 hybrid_clock.cc:648] HybridClock initialized: now 1786189678222277 us; error 37 us; skew 500 ppm
I20260808 11:47:58.222649 5226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:47:58 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:47:58.225842 5226 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:47:58.227293 5226 webserver.cc:460] Webserver started at http://127.13.158.67:37945/ using document root <none> and password file <none>
I20260808 11:47:58.227969 5226 fs_manager.cc:359] Metadata directory not provided
I20260808 11:47:58.228081 5226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:47:58.233304 5226 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:47:58.236101 5249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:58.237565 5226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:47:58.237753 5226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "6a07ea4184fa421f976f80be31a78f53"
format_stamp: "Formatted at 2026-08-08 11:47:35 on dist-test-slave-39l8"
server_key: "c3871ce5ee5f44a4fe89a8ccf190d89a"
server_key_iv: "d13cfad7f8c8853064d10a0148710410"
server_key_version: "encryptionkey@0"
I20260808 11:47:58.238692 5226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:47:58.253790 5226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:47:58.258070 5226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:47:58.258339 5226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:47:58.259464 5226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:47:58.260788 5226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:47:58.260915 5226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:58.261039 5226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:47:58.261130 5226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:47:58.295326 5226 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:37611
I20260808 11:47:58.295409 5412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:37611 every 8 connection(s)
I20260808 11:47:58.296738 5226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:47:58.299484 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 5226
Aug 08 11:47:58 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:47:58.318826 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.300577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42800 (local address 127.13.158.126:33657)
0808 11:47:58.303045 (+ 2468us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.303049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.304499 (+ 1450us) server_negotiation.cc:407] Connection header received
0808 11:47:58.304690 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.304694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.304771 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.304879 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:58.306615 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.307590 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.309029 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.309537 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.309709 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.312544 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:58.312548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:58.312552 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:58.312583 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:58.315769 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.316632 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.316637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.316639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.316831 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.317227 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.317233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.317237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.317629 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:58.317889 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.318214 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.318398 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":2374,"spinlock_wait_cycles":3328,"thread_start_us":2258,"threads_started":1}
I20260808 11:47:58.320298 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.299166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:39365 (local address 127.13.158.126:33657)
0808 11:47:58.299343 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.299347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.300858 (+ 1511us) server_negotiation.cc:407] Connection header received
0808 11:47:58.304375 (+ 3517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.304380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.304500 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.304604 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:58.306206 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.307167 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.308927 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.309495 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.309590 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.313273 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:58.313280 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:58.313286 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:58.313326 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:58.316662 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.317575 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.317578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.317580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.317722 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.318232 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.318237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.318240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.318586 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:58.318777 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.319727 (+ 950us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.319919 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":896}
I20260808 11:47:58.321544 5418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.299721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:39365)
0808 11:47:58.300652 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.300709 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:58.304037 (+ 3328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:58.304875 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:58.304886 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:58.305470 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:47:58.306002 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.306018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.307373 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.307379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.308773 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.308786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.309735 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.309744 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.310020 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.310826 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:47:58.310848 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:47:58.313053 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:47:58.316810 (+ 3757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:58.316819 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:58.316837 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:58.317456 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:58.317851 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:47:58.317856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:47:58.317859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:47:58.318105 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:47:58.318819 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:47:58.318827 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:47:58.319317 (+ 490us) client_negotiation.cc:769] Sending connection context
0808 11:47:58.319665 (+ 348us) client_negotiation.cc:240] Negotiation successful
0808 11:47:58.320193 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":375,"threads_started":1}
I20260808 11:47:58.323042 5413 heartbeater.cc:347] Connected to a master server at 127.13.158.126:33657
I20260808 11:47:58.323333 5413 heartbeater.cc:464] Registering TS with master...
I20260808 11:47:58.323992 5413 heartbeater.cc:511] Master 127.13.158.126:33657 requested a full tablet report, sending...
I20260808 11:47:58.325399 4748 ts_manager.cc:194] Registered new tserver with Master: 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
I20260808 11:47:58.326180 4748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:39365
I20260808 11:47:58.337893 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:47:58.344867 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.338869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.13.158.126:33657)
0808 11:47:58.339096 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.339105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.339517 (+ 412us) server_negotiation.cc:407] Connection header received
0808 11:47:58.339717 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.339725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.339883 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.340004 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:47:58.340449 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.341183 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.342908 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.343352 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.343502 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.344013 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:47:58.344213 (+ 200us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 11:47:58.344605 (+ 392us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42814: token expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 11:47:58.344959 5018 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42814: token expired
W20260808 11:47:58.345470 4632 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.158.126:33657)
I20260808 11:47:58.358686 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.345927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.13.158.126:33657)
0808 11:47:58.346090 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.346094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.346109 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:47:58.346237 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.346240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.346289 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.346362 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:58.347791 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.348458 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.349706 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.350074 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.350163 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.352764 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:58.352770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:58.352775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:58.352810 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:58.356023 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.356742 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.356747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.356751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.356880 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.357223 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.357228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.357232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.357619 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:58.357891 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.358084 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.358354 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:47:58.361909 4748 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:47:58.369709 4748 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:47:58.393985 5016 heartbeater.cc:503] Master 127.13.158.126:33657 was elected leader, sending a full tablet report...
I20260808 11:47:58.394371 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.380865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:46095 (local address 127.0.0.1:53824)
0808 11:47:58.381809 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.381847 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:58.382019 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:58.383899 (+ 1880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:58.383910 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:58.383950 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:58.384473 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.384489 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.387330 (+ 2841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.387339 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.390433 (+ 3094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.390456 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.393122 (+ 2666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.393131 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.393436 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.393515 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 11:47:58.393643 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 11:47:58.393895 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":1289,"threads_started":1}
I20260808 11:47:58.395714 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.380914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53824 (local address 127.13.158.66:46095)
0808 11:47:58.383142 (+ 2228us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.383155 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.383191 (+ 36us) server_negotiation.cc:407] Connection header received
0808 11:47:58.383309 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.383319 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.383609 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.384753 (+ 1144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:58.385037 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.387117 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.390721 (+ 3604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.392738 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.394908 (+ 2170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.395138 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.395218 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.395394 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2054,"thread_start_us":1963,"threads_started":1}
I20260808 11:47:58.400955 5127 tablet_service.cc:1467] Processing CreateTablet for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0 (DEFAULT_TABLE table=test-table [id=dc79d1b466534583a0aeb8adbe79160c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:58.402393 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.381772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58760 (local address 127.13.158.65:42487)
0808 11:47:58.394602 (+ 12830us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.394613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.394645 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:47:58.394806 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.394816 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.395059 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.396098 (+ 1039us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:58.396306 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.397493 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.399978 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.401391 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.401536 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.401874 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.401947 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.402103 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12633,"thread_start_us":13419,"threads_started":1}
I20260808 11:47:58.402392 5435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.381014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58760)
0808 11:47:58.382626 (+ 1612us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.382642 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:58.382783 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:58.395455 (+ 12672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:58.395461 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:58.395489 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:58.395898 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.395914 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.397651 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.397659 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.399783 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.399801 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.401612 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.401619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.401809 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.401839 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:47:58.401919 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 11:47:58.402044 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1493,"thread_start_us":1363,"threads_started":1}
I20260808 11:47:58.402488 5127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:58.402988 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.382925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37611 (local address 127.0.0.1:53526)
0808 11:47:58.394595 (+ 11670us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.394614 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:58.394746 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:58.396696 (+ 1950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:58.396702 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:58.396731 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:58.396947 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.396959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.398796 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.398800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.400536 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.400570 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.402241 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.402247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.402430 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.402470 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 11:47:58.402571 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 11:47:58.402756 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11582,"thread_start_us":13750,"threads_started":1}
I20260808 11:47:58.403349 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.383407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.13.158.67:37611)
0808 11:47:58.396046 (+ 12639us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.396056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.396091 (+ 35us) server_negotiation.cc:407] Connection header received
0808 11:47:58.396190 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.396199 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.396415 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.397129 (+ 714us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:58.397407 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.398593 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.400724 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.401838 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.401954 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.402149 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.402841 (+ 692us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.402973 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12437,"thread_start_us":12874,"threads_started":1}
I20260808 11:47:58.404408 4928 tablet_service.cc:1467] Processing CreateTablet for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0 (DEFAULT_TABLE table=test-table [id=dc79d1b466534583a0aeb8adbe79160c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:58.405575 4928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:58.407018 5318 tablet_service.cc:1467] Processing CreateTablet for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0 (DEFAULT_TABLE table=test-table [id=dc79d1b466534583a0aeb8adbe79160c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:47:58.408293 5318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:47:58.414880 5439 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Bootstrap starting.
I20260808 11:47:58.418047 5439 tablet_bootstrap.cc:654] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:58.418521 5441 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Bootstrap starting.
I20260808 11:47:58.419384 5439 log.cc:824] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:58.422367 5441 tablet_bootstrap.cc:654] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:58.422499 5439 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: No bootstrap required, opened a new log
I20260808 11:47:58.422775 5439 ts_tablet_manager.cc:1397] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260808 11:47:58.423635 5441 log.cc:824] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:58.423909 5443 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Bootstrap starting.
I20260808 11:47:58.426999 5443 tablet_bootstrap.cc:654] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Neither blocks nor log segments found. Creating new log.
I20260808 11:47:58.427204 5441 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: No bootstrap required, opened a new log
I20260808 11:47:58.427444 5441 ts_tablet_manager.cc:1397] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 11:47:58.428105 5443 log.cc:824] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Log is configured to *not* fsync() on all Append() calls
I20260808 11:47:58.430415 5443 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: No bootstrap required, opened a new log
I20260808 11:47:58.430673 5443 ts_tablet_manager.cc:1397] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 11:47:58.433187 5439 raft_consensus.cc:348] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:58.433602 5439 raft_consensus.cc:374] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:58.433714 5439 raft_consensus.cc:729] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43ff67b9a9a4f56ab9f8fce8e7b4c65, State: Initialized, Role: FOLLOWER
I20260808 11:47:58.434296 5439 consensus_queue.cc:260] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:58.436004 5439 ts_tablet_manager.cc:1428] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260808 11:47:58.438901 5441 raft_consensus.cc:348] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:58.439392 5441 raft_consensus.cc:374] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:58.439510 5441 raft_consensus.cc:729] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b36bdbb96cd4018a168b925706c178f, State: Initialized, Role: FOLLOWER
I20260808 11:47:58.439653 5443 raft_consensus.cc:348] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:58.440042 5443 raft_consensus.cc:374] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:47:58.440161 5443 raft_consensus.cc:729] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a07ea4184fa421f976f80be31a78f53, State: Initialized, Role: FOLLOWER
I20260808 11:47:58.440135 5441 consensus_queue.cc:260] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:58.440927 5443 consensus_queue.cc:260] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:58.441509 5219 heartbeater.cc:503] Master 127.13.158.126:33657 was elected leader, sending a full tablet report...
I20260808 11:47:58.442119 5441 ts_tablet_manager.cc:1428] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260808 11:47:58.442183 5413 heartbeater.cc:503] Master 127.13.158.126:33657 was elected leader, sending a full tablet report...
I20260808 11:47:58.443545 5443 ts_tablet_manager.cc:1428] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Time spent starting tablet: real 0.013s user 0.002s sys 0.009s
W20260808 11:47:58.548350 5414 tablet.cc:2369] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:47:58.569744 5221 tablet.cc:2369] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:58.585876 5449 raft_consensus.cc:482] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:47:58.586221 5449 raft_consensus.cc:504] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:58.588027 5449 leader_election.cc:288] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f43ff67b9a9a4f56ab9f8fce8e7b4c65 (127.13.158.65:42487), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
I20260808 11:47:58.595271 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.588496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:41231 (local address 127.13.158.65:42487)
0808 11:47:58.588711 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.588716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.589645 (+ 929us) server_negotiation.cc:407] Connection header received
0808 11:47:58.589839 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.589847 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.589932 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.590020 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:58.590763 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.591556 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.593408 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.594263 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.594381 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.594431 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.594874 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.595000 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:47:58.595343 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.588584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.13.158.66:41231)
0808 11:47:58.589481 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.589506 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:58.589685 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:58.590055 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:58.590059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:58.590083 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:58.590571 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.590581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.591693 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.591698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.593239 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.593251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.594488 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.594495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.594666 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.594702 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:47:58.594818 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 11:47:58.594915 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":46,"thread_start_us":435,"threads_started":1}
I20260808 11:47:58.595516 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.588573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:51371 (local address 127.13.158.67:37611)
0808 11:47:58.588799 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.588808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.589831 (+ 1023us) server_negotiation.cc:407] Connection header received
0808 11:47:58.590020 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.590025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.590102 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.590208 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:47:58.591006 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.591792 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.593615 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.594470 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.594634 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.594739 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.595129 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.595265 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:47:58.595896 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.588584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37611 (local address 127.13.158.66:51371)
0808 11:47:58.589689 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.589709 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:47:58.589864 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:47:58.590250 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:47:58.590253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:47:58.590297 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:47:58.590845 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.590860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.591933 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.591937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.593457 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:47:58.593473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.594681 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.594686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:47:58.594948 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.594981 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:47:58.595083 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 11:47:58.595204 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"mutex_wait_us":294,"spinlock_wait_cycles":8576,"thread_start_us":468,"threads_started":1}
I20260808 11:47:58.596536 4955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aaeb6dcbbbd45bbb38ae942b40827c0" candidate_uuid: "2b36bdbb96cd4018a168b925706c178f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" is_pre_election: true
I20260808 11:47:58.597029 4955 raft_consensus.cc:2454] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b36bdbb96cd4018a168b925706c178f in term 0.
I20260808 11:47:58.596944 5344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aaeb6dcbbbd45bbb38ae942b40827c0" candidate_uuid: "2b36bdbb96cd4018a168b925706c178f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a07ea4184fa421f976f80be31a78f53" is_pre_election: true
I20260808 11:47:58.597453 5344 raft_consensus.cc:2454] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b36bdbb96cd4018a168b925706c178f in term 0.
I20260808 11:47:58.597703 5054 leader_election.cc:302] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b36bdbb96cd4018a168b925706c178f, f43ff67b9a9a4f56ab9f8fce8e7b4c65; no voters:
I20260808 11:47:58.598097 5449 raft_consensus.cc:2781] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:47:58.598258 5449 raft_consensus.cc:482] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:47:58.598387 5449 raft_consensus.cc:3037] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:58.600724 5449 raft_consensus.cc:504] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:47:58.601365 5449 leader_election.cc:288] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [CANDIDATE]: Term 1 election: Requested vote from peers f43ff67b9a9a4f56ab9f8fce8e7b4c65 (127.13.158.65:42487), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
I20260808 11:47:58.601816 4955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aaeb6dcbbbd45bbb38ae942b40827c0" candidate_uuid: "2b36bdbb96cd4018a168b925706c178f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65"
I20260808 11:47:58.601850 5344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aaeb6dcbbbd45bbb38ae942b40827c0" candidate_uuid: "2b36bdbb96cd4018a168b925706c178f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a07ea4184fa421f976f80be31a78f53"
I20260808 11:47:58.602005 4955 raft_consensus.cc:3037] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:58.602026 5344 raft_consensus.cc:3037] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:47:58.604424 5344 raft_consensus.cc:2454] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b36bdbb96cd4018a168b925706c178f in term 1.
I20260808 11:47:58.604611 4955 raft_consensus.cc:2454] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b36bdbb96cd4018a168b925706c178f in term 1.
I20260808 11:47:58.604866 5052 leader_election.cc:302] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b36bdbb96cd4018a168b925706c178f, 6a07ea4184fa421f976f80be31a78f53; no voters:
I20260808 11:47:58.605166 5449 raft_consensus.cc:2781] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:47:58.605633 5449 raft_consensus.cc:686] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 1 LEADER]: Becoming Leader. State: Replica: 2b36bdbb96cd4018a168b925706c178f, State: Running, Role: LEADER
I20260808 11:47:58.606074 5449 consensus_queue.cc:237] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
W20260808 11:47:58.606513 5017 tablet.cc:2369] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:47:58.610554 4748 catalog_manager.cc:5495] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b36bdbb96cd4018a168b925706c178f (127.13.158.66). New cstate: current_term: 1 leader_uuid: "2b36bdbb96cd4018a168b925706c178f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } health_report { overall_health: UNKNOWN } } }
I20260808 11:47:58.630973 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.625442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53826 (local address 127.13.158.66:46095)
0808 11:47:58.625725 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.625733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.625846 (+ 113us) server_negotiation.cc:407] Connection header received
0808 11:47:58.626107 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.626111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.626197 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.626358 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:47:58.626775 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.627463 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.628814 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.629217 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.629346 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.629877 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:47:58.630344 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:47:58.630538 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.630588 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.630678 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260808 11:47:58.638731 5344 raft_consensus.cc:1261] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 1 FOLLOWER]: Refusing update from remote peer 2b36bdbb96cd4018a168b925706c178f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:58.638906 4955 raft_consensus.cc:1261] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Refusing update from remote peer 2b36bdbb96cd4018a168b925706c178f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:47:58.639772 5453 consensus_queue.cc:1035] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:47:58.640204 5449 consensus_queue.cc:1035] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:47:58.649780 5457 mvcc.cc:205] Tried to move back new op lower bound from 7316232923694690304 to 7316232923574792192. Current Snapshot: MvccSnapshot[applied={T|T < 7316232923694690304}]
I20260808 11:47:58.653002 5460 mvcc.cc:205] Tried to move back new op lower bound from 7316232923694690304 to 7316232923574792192. Current Snapshot: MvccSnapshot[applied={T|T < 7316232923694690304}]
I20260808 11:47:58.669580 4747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:47:58.669869 4747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:47:58.673657 4747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Sending DeleteTablet for 3 replicas of tablet 6aaeb6dcbbbd45bbb38ae942b40827c0
I20260808 11:47:58.674784 4928 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:58 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:58760
I20260808 11:47:58.674883 5127 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:58 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53824
I20260808 11:47:58.675017 5318 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:47:58 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53526
I20260808 11:47:58.676287 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 4798
I20260808 11:47:58.687266 5471 tablet_replica.cc:330] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: stopping tablet replica
I20260808 11:47:58.687844 5471 raft_consensus.cc:2229] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:47:58.688627 5471 raft_consensus.cc:2258] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:58.690902 5471 ts_tablet_manager.cc:1905] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:58.698537 5471 ts_tablet_manager.cc:1918] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:58.698799 5471 log.cc:1196] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/6aaeb6dcbbbd45bbb38ae942b40827c0
I20260808 11:47:58.699455 5471 ts_tablet_manager.cc:1939] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 2b36bdbb96cd4018a168b925706c178f: Deleting consensus metadata
I20260808 11:47:58.701538 4728 catalog_manager.cc:4841] TS 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095): tablet 6aaeb6dcbbbd45bbb38ae942b40827c0 (table test-table [id=dc79d1b466534583a0aeb8adbe79160c]) successfully deleted
I20260808 11:47:58.707767 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.677971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.13.158.65:42487)
0808 11:47:58.678178 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.678185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.678206 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:47:58.678286 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.678292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.678382 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.678487 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:58.681203 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.682782 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.686593 (+ 3811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.687525 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.687681 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.697017 (+ 9336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:58.697040 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:58.697064 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:58.697121 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:58.702760 (+ 5639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.704187 (+ 1427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.704198 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.704203 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.704654 (+ 451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.705193 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.705198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.705202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.705692 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:58.706090 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.707099 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.707337 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:47:58.709484 5009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:58.714365 5472 tablet_replica.cc:330] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: stopping tablet replica
I20260808 11:47:58.714847 5472 raft_consensus.cc:2229] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:47:58.715510 5472 raft_consensus.cc:2258] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:58.717527 5472 ts_tablet_manager.cc:1905] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:47:58.724255 5472 ts_tablet_manager.cc:1918] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:47:58.724525 5472 log.cc:1196] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/6aaeb6dcbbbd45bbb38ae942b40827c0
I20260808 11:47:58.725206 5472 ts_tablet_manager.cc:1939] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P 6a07ea4184fa421f976f80be31a78f53: Deleting consensus metadata
I20260808 11:47:58.726934 4728 catalog_manager.cc:4841] TS 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611): tablet 6aaeb6dcbbbd45bbb38ae942b40827c0 (table test-table [id=dc79d1b466534583a0aeb8adbe79160c]) successfully deleted
I20260808 11:47:58.881026 5469 tablet_replica.cc:330] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: stopping tablet replica
I20260808 11:47:58.881649 5469 raft_consensus.cc:2229] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:47:58.882238 5469 raft_consensus.cc:2258] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:47:58.883204 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 4798
W20260808 11:47:58.919730 4729 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.158.65:42487 (error 108)
I20260808 11:47:58.920625 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 5024
I20260808 11:47:58.924098 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.923272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58780)
0808 11:47:58.923753 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.923885 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":238,"threads_started":1}
W20260808 11:47:58.924410 4729 catalog_manager.cc:4584] TS f43ff67b9a9a4f56ab9f8fce8e7b4c65 (127.13.158.65:42487): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6aaeb6dcbbbd45bbb38ae942b40827c0: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
I20260808 11:47:58.938413 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.921592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.13.158.66:46095)
0808 11:47:58.921845 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:58.921854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:58.921881 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:47:58.922083 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:58.922092 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:58.922205 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:58.922342 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:58.924446 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.925426 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.927476 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:58.927928 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:58.928022 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:58.931494 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:58.931506 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:58.931524 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:58.931561 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:58.935499 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.936204 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.936213 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.936219 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.936422 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:58.936829 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:58.936835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:58.936839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:58.937289 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:58.937636 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:58.937796 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 11:47:58.938011 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 11:47:58.939147 5212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:58.978027 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.977513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58784)
0808 11:47:58.977747 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.977857 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 11:47:58.979225 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:58.978806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58796)
0808 11:47:58.978984 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:58.979075 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 11:47:59.039976 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:59.039407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58802)
0808 11:47:59.039655 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:59.039765 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 11:47:59.041337 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:59.040931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58810)
0808 11:47:59.041088 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:59.041183 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 11:47:59.053577 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 5024
I20260808 11:47:59.086992 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 5226
I20260808 11:47:59.101104 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:59.087749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53542 (local address 127.13.158.67:37611)
0808 11:47:59.087951 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:47:59.087957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:47:59.088057 (+ 100us) server_negotiation.cc:407] Connection header received
0808 11:47:59.088292 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:47:59.088297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:47:59.088376 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:47:59.088488 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:47:59.089892 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:59.090625 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:59.092082 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:47:59.092488 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:47:59.092629 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:47:59.095489 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:47:59.095505 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:47:59.095522 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:47:59.095558 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:47:59.098087 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:59.098850 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:59.098863 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:59.098866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:59.099024 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:47:59.099448 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:47:59.099459 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:47:59.099463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:47:59.099840 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:47:59.100185 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0808 11:47:59.100519 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 11:47:59.100747 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:47:59.101768 5405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:59.121956 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:59.121524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58816)
0808 11:47:59.121720 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:59.121809 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 11:47:59.123090 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:59.122619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58820)
0808 11:47:59.122884 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:59.122955 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260808 11:47:59.191990 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 5226
I20260808 11:47:59.216730 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 4696
I20260808 11:47:59.217938 4789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:47:59.330238 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:47:59.329702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:42487 (local address 127.0.0.1:58832)
0808 11:47:59.329909 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 11:47:59.330046 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:42487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 11:47:59.330334 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 4696
I20260808 11:48:19.359287 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:33657
--webserver_interface=127.13.158.126
--webserver_port=42973
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:19.714653 5481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:19.714998 5481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:19.715171 5481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:19.722229 5481 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 11:48:19.722365 5481 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 11:48:19.722466 5481 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 11:48:19.722572 5481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:48:19.722678 5481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:19.722808 5481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:19.722898 5481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:48:19.722980 5481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:48:19.732628 5481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=42973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:19.733886 5481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:19.735320 5481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:19.747118 5493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:19.747136 5489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:19.747169 5490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:19.748193 5481 server_base.cc:1034] running on GCE node
I20260808 11:48:19.748983 5481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:19.750277 5481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:19.751518 5481 hybrid_clock.cc:648] HybridClock initialized: now 1786189699751445 us; error 46 us; skew 500 ppm
I20260808 11:48:19.751835 5481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:19 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:19.755062 5481 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:48:19.756438 5481 webserver.cc:460] Webserver started at http://127.13.158.126:42973/ using document root <none> and password file <none>
I20260808 11:48:19.757131 5481 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:19.757274 5481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:19.762518 5481 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:48:19.765321 5506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:19.766606 5481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:48:19.766824 5481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf"
format_stamp: "Formatted at 2026-08-08 11:47:33 on dist-test-slave-39l8"
server_key: "eb1655c8801d46ef4a1de3222e245e9b"
server_key_iv: "15dd8a25b37a0f0605bf5757a4e1faf2"
server_key_version: "encryptionkey@0"
I20260808 11:48:19.767750 5481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:19.779953 5481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:19.783632 5481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:19.783910 5481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:19.802875 5481 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:33657
I20260808 11:48:19.802927 5583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:33657 every 8 connection(s)
I20260808 11:48:19.805362 5481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:48:19.811813 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 5481
I20260808 11:48:19.812574 5584 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:48:19.812973 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:42487
--local_ip_for_outbound_sockets=127.13.158.65
--tserver_master_addrs=127.13.158.126:33657
--webserver_port=38541
--webserver_interface=127.13.158.65
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:48:19.817619 5584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Bootstrap starting.
I20260808 11:48:19.840224 5584 log.cc:824] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:19.856020 5584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:48:19.856559 5584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Bootstrap complete.
I20260808 11:48:19.865795 5584 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:48:19.866643 5584 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b148f85e5240e19bc4a4e21ae3ebcf, State: Initialized, Role: FOLLOWER
I20260808 11:48:19.867441 5584 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:48:19.867676 5584 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:48:19.867799 5584 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:48:19.867996 5584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 2 FOLLOWER]: Advancing to term 3
I20260808 11:48:19.872205 5584 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:48:19.872763 5584 leader_election.cc:302] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b148f85e5240e19bc4a4e21ae3ebcf; no voters:
I20260808 11:48:19.873452 5584 leader_election.cc:288] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 11:48:19.873652 5590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 3 FOLLOWER]: Leader election won for term 3
I20260808 11:48:19.874785 5590 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [term 3 LEADER]: Becoming Leader. State: Replica: c3b148f85e5240e19bc4a4e21ae3ebcf, State: Running, Role: LEADER
I20260808 11:48:19.875367 5590 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } }
I20260808 11:48:19.875533 5584 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:48:19.878002 5593 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b148f85e5240e19bc4a4e21ae3ebcf. Latest consensus state: current_term: 3 leader_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } } }
I20260808 11:48:19.878444 5593 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:19.879105 5591 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b148f85e5240e19bc4a4e21ae3ebcf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 33657 } } }
I20260808 11:48:19.879446 5591 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:19.889302 5598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:48:19.899991 5598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:48:19.902237 5598 catalog_manager.cc:1259] Loaded cluster ID: 32a6626ddf0d4af2b969288302a8648c
I20260808 11:48:19.902810 5598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:48:19.908161 5598 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:48:19.910487 5598 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Loaded TSK: 0
I20260808 11:48:19.910800 5598 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c3b148f85e5240e19bc4a4e21ae3ebcf: Loaded TSK: 1
F20260808 11:48:19.912304 5598 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff7c63c29ad google::LogMessage::Fail() at ??:0
@ 0x7ff7c63c588a google::LogMessage::SendToLog() at ??:0
@ 0x7ff7c63c24de google::LogMessage::Flush() at ??:0
@ 0x7ff7c63c3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff7ca1267d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff7ca11573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff7dbdf11c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff7dbdf0783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff7dbe7c2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7ff7dbe7c0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7ff7dbec65e9 std::function<>::operator()() at ??:0
@ 0x7ff7dbdf4cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7ff7dbdf2c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff7dbe7a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7ff7dbe7a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff7c900e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff7c901191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7ff7c9011799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff7c8feceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff7cc2c66db start_thread at ??:0
@ 0x7ff7c525c61f clone at ??:0
W20260808 11:48:20.231258 5586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:20.231544 5586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:20.231642 5586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:20.238876 5586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:20.238981 5586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:20.239099 5586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:48:20.248610 5586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:42487
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=38541
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:20.249851 5586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:20.251212 5586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:20.264508 5648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:20.264449 5653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:20.264550 5649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:20.265745 5586 server_base.cc:1034] running on GCE node
I20260808 11:48:20.266337 5586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:20.267300 5586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:20.268534 5586 hybrid_clock.cc:648] HybridClock initialized: now 1786189700268490 us; error 51 us; skew 500 ppm
I20260808 11:48:20.268865 5586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:20 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189700, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:20.272012 5586 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:48:20.273370 5586 webserver.cc:460] Webserver started at http://127.13.158.65:38541/ using document root <none> and password file <none>
I20260808 11:48:20.274087 5586 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:20.274238 5586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:20.279481 5586 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:20.282339 5664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:20.283668 5586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:48:20.283866 5586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "204657dc50e393c9ddf8d8ca8dc11222"
server_key_iv: "d3ecc3b5b102ede9130eb0fb38c402ce"
server_key_version: "encryptionkey@0"
I20260808 11:48:20.284807 5586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:20.308140 5586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:20.312034 5586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:20.312309 5586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:20.313287 5586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:20.315061 5675 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 11:48:20.320568 5586 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 11:48:20.320724 5586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260808 11:48:20.320868 5586 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 11:48:20.323887 5586 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 11:48:20.323994 5586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.002s
I20260808 11:48:20.324301 5675 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Bootstrap starting.
I20260808 11:48:20.351879 5675 log.cc:824] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:20.364459 5586 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:42487
I20260808 11:48:20.364481 5834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:42487 every 8 connection(s)
I20260808 11:48:20.365736 5586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:48:20.365747 5675 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:48:20.366343 5675 tablet_bootstrap.cc:492] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Bootstrap complete.
I20260808 11:48:20.368017 5675 ts_tablet_manager.cc:1397] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Time spent bootstrapping tablet: real 0.044s user 0.029s sys 0.012s
I20260808 11:48:20.370633 5838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:20.368525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:53367)
0808 11:48:20.369403 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:20.369734 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":398,"threads_started":1}
W20260808 11:48:20.371354 5836 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:33657 (0 consecutive failures): Network error: Failed to ping master at 127.13.158.126:33657: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
I20260808 11:48:20.372192 5838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:20.371847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:40139)
0808 11:48:20.372010 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:20.372083 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 11:48:20.373076 5838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:20.372713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:58431)
0808 11:48:20.372855 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:20.372947 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:48:20.373327 5836 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:48:20.375178 5675 raft_consensus.cc:348] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:48:20.376056 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 5586
I20260808 11:48:20.376052 5675 raft_consensus.cc:729] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43ff67b9a9a4f56ab9f8fce8e7b4c65, State: Initialized, Role: FOLLOWER
I20260808 11:48:20.376717 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:46095
--local_ip_for_outbound_sockets=127.13.158.66
--tserver_master_addrs=127.13.158.126:33657
--webserver_port=33597
--webserver_interface=127.13.158.66
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:48:20.376648 5675 consensus_queue.cc:260] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:48:20.378113 5675 ts_tablet_manager.cc:1428] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260808 11:48:20.378284 5838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:20.377876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:58863)
0808 11:48:20.378046 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:20.378138 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 11:48:20.379163 5838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:20.378859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:46607)
0808 11:48:20.378976 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:20.379035 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 11:48:20.715543 5840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:20.715798 5840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:20.715889 5840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:20.722762 5840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:20.722872 5840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:20.722978 5840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:48:20.732223 5840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:46095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=33597
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:20.733327 5840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:20.734550 5840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:20.747233 5849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:20.747205 5850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:20.747154 5854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:20.748636 5840 server_base.cc:1034] running on GCE node
I20260808 11:48:20.749217 5840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:20.750077 5840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:20.751299 5840 hybrid_clock.cc:648] HybridClock initialized: now 1786189700751245 us; error 69 us; skew 500 ppm
I20260808 11:48:20.751600 5840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:20 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189700, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:20.754576 5840 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:48:20.755991 5840 webserver.cc:460] Webserver started at http://127.13.158.66:33597/ using document root <none> and password file <none>
I20260808 11:48:20.756652 5840 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:20.756803 5840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:20.761855 5840 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:48:20.764437 5866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:20.765692 5840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:48:20.765868 5840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "2b36bdbb96cd4018a168b925706c178f"
format_stamp: "Formatted at 2026-08-08 11:47:34 on dist-test-slave-39l8"
server_key: "bea281189b602658c72321907367bfad"
server_key_iv: "ea7547bb8984386679ff4d0bd8cc6ef6"
server_key_version: "encryptionkey@0"
I20260808 11:48:20.766906 5840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:20.780076 5840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:20.783860 5840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:20.784132 5840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:20.785036 5840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:20.786309 5840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:20.786428 5840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:20.786562 5840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:20.786653 5840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:20.822117 5840 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:46095
I20260808 11:48:20.822206 6024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:46095 every 8 connection(s)
I20260808 11:48:20.823614 5840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:48:20.826340 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 5840
I20260808 11:48:20.827039 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:37611
--local_ip_for_outbound_sockets=127.13.158.67
--tserver_master_addrs=127.13.158.126:33657
--webserver_port=37945
--webserver_interface=127.13.158.67
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:48:20.828531 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:20.826542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:38971)
0808 11:48:20.827417 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:20.827716 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":393,"threads_started":1}
W20260808 11:48:20.829177 6025 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:33657 (0 consecutive failures): Network error: Failed to ping master at 127.13.158.126:33657: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
I20260808 11:48:20.830021 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:20.829722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:52185)
0808 11:48:20.829861 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:20.829925 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 11:48:20.831000 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:20.830538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:37261)
0808 11:48:20.830735 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:20.830837 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 11:48:20.831254 6025 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:48:21.165192 6030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:21.165482 6030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:21.165678 6030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:21.172973 6030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:21.173108 6030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:21.173300 6030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:48:21.183014 6030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:37611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=37945
--tserver_master_addrs=127.13.158.126:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:21.184296 6030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:21.185578 6030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:21.198690 6038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:21.198869 6039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:21.199272 6041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:21.199889 6030 server_base.cc:1034] running on GCE node
I20260808 11:48:21.200479 6030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:21.201467 6030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:21.202696 6030 hybrid_clock.cc:648] HybridClock initialized: now 1786189701202645 us; error 62 us; skew 500 ppm
I20260808 11:48:21.203056 6030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:21 dist-test-slave-39l8 krb5kdc[3871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189701, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:21.206185 6030 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:48:21.207661 6030 webserver.cc:460] Webserver started at http://127.13.158.67:37945/ using document root <none> and password file <none>
I20260808 11:48:21.208348 6030 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:21.208503 6030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:21.213847 6030 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 11:48:21.216619 6052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:21.217918 6030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:48:21.218101 6030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "6a07ea4184fa421f976f80be31a78f53"
format_stamp: "Formatted at 2026-08-08 11:47:35 on dist-test-slave-39l8"
server_key: "c3871ce5ee5f44a4fe89a8ccf190d89a"
server_key_iv: "d13cfad7f8c8853064d10a0148710410"
server_key_version: "encryptionkey@0"
I20260808 11:48:21.219162 6030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:21.248639 6030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:21.252624 6030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:21.252915 6030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:21.253930 6030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:21.255367 6030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:21.255494 6030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:21.255620 6030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:21.255720 6030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:21.290175 6030 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:37611
I20260808 11:48:21.290264 6212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:37611 every 8 connection(s)
I20260808 11:48:21.291677 6030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:48:21.296273 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 6030
I20260808 11:48:21.296816 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:21.294631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:40895)
0808 11:48:21.295393 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:21.295653 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":277,"threads_started":1}
W20260808 11:48:21.298414 6213 heartbeater.cc:644] Failed to heartbeat to 127.13.158.126:33657 (0 consecutive failures): Network error: Failed to ping master at 127.13.158.126:33657: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
I20260808 11:48:21.299302 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:21.298955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:34021)
0808 11:48:21.299136 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:21.299196 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260808 11:48:21.300174 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:21.299824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:51893)
0808 11:48:21.299986 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:21.300059 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 11:48:21.300426 6213 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:48:21.380688 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:21.379944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:56643)
0808 11:48:21.380389 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:21.380482 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":217,"threads_started":1}
I20260808 11:48:21.832528 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:21.831763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:47303)
0808 11:48:21.832203 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:21.832306 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":224,"threads_started":1}
I20260808 11:48:22.162974 6220 raft_consensus.cc:482] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:22.163302 6220 raft_consensus.cc:504] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:48:22.164943 6220 leader_election.cc:288] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
Aug 08 11:48:22 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189700, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
Aug 08 11:48:22 dist-test-slave-39l8 krb5kdc[3871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189700, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:48:22.187139 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:22.166512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:35967 (local address 127.13.158.67:37611)
0808 11:48:22.167129 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:22.167140 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:22.167172 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:48:22.168562 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:22.168570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:22.168940 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:22.169287 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:22.171663 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:22.172834 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:22.174586 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:22.175321 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:22.175528 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:22.179626 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:22.179637 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:22.179658 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:22.179721 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:22.183488 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:22.184310 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:22.184316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:22.184321 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:22.184470 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:22.184885 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:22.184889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:22.184891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:22.185436 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:22.185629 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:22.186323 (+ 694us) server_negotiation.cc:299] Negotiation successful
0808 11:48:22.186726 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":384,"threads_started":1}
I20260808 11:48:22.187157 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:22.166128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37611 (local address 127.13.158.65:35967)
0808 11:48:22.166837 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:22.166856 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:22.166969 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:22.169450 (+ 2481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:22.169462 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:22.170660 (+ 1198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:22.171450 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:22.171472 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:22.173044 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:22.173048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:22.174440 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:22.174453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:22.175549 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:22.175556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:22.175861 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:22.176716 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:22.176733 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:22.179290 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:22.183630 (+ 4340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:22.183634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:22.183636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:22.184108 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:22.184591 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:22.184599 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:22.184602 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:22.184759 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:22.185577 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:22.185582 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:22.185825 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 11:48:22.186094 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 11:48:22.186618 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":69,"spinlock_wait_cycles":20096,"thread_start_us":469,"threads_started":1}
I20260808 11:48:22.187171 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:22.165580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:50065 (local address 127.13.158.66:46095)
0808 11:48:22.166261 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:22.166277 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:22.166327 (+ 50us) server_negotiation.cc:407] Connection header received
0808 11:48:22.168548 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:22.168556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:22.168941 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:22.169266 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:22.171662 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:22.172834 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:22.174669 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:22.175346 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:22.175547 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:22.179865 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:22.179871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:22.179882 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:22.179917 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:22.183092 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:22.184048 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:22.184058 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:22.184066 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:22.184335 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:22.184796 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:22.184801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:22.184805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:22.185342 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:22.185549 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:22.186350 (+ 801us) server_negotiation.cc:299] Negotiation successful
0808 11:48:22.186748 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":322,"threads_started":1}
I20260808 11:48:22.187232 6223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:22.165238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:46095 (local address 127.13.158.65:50065)
0808 11:48:22.165887 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:22.165944 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:22.166599 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:22.169450 (+ 2851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:22.169462 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:22.170668 (+ 1206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:22.171450 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:22.171472 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:22.173027 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:22.173032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:22.174467 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:22.174479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:22.175574 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:22.175580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:22.175859 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:22.176645 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:22.176672 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:22.179618 (+ 2946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:22.183249 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:22.183260 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:22.183278 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:22.183891 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:22.184452 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:22.184458 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:22.184462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:22.184670 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:22.185486 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:22.185496 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:22.185825 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 11:48:22.186095 (+ 270us) client_negotiation.cc:240] Negotiation successful
0808 11:48:22.186794 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":364,"spinlock_wait_cycles":23296,"thread_start_us":313,"threads_started":1}
W20260808 11:48:22.190495 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
W20260808 11:48:22.190861 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
I20260808 11:48:22.190984 5668 leader_election.cc:302] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f43ff67b9a9a4f56ab9f8fce8e7b4c65; no voters: 2b36bdbb96cd4018a168b925706c178f, 6a07ea4184fa421f976f80be31a78f53
I20260808 11:48:22.191341 6220 raft_consensus.cc:2726] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:48:22.301991 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:22.301045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:48443)
0808 11:48:22.301572 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:22.301745 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":292,"threads_started":1}
I20260808 11:48:22.381897 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:22.381498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:35295)
0808 11:48:22.381665 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:22.381755 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 11:48:22.834182 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:22.833498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:43701)
0808 11:48:22.833932 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:22.834012 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":243,"threads_started":1}
I20260808 11:48:23.303833 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:23.302996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:39747)
0808 11:48:23.303561 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:23.303672 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":290,"threads_started":1}
I20260808 11:48:23.383570 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:23.382802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:49923)
0808 11:48:23.383296 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:23.383385 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":218,"threads_started":1}
I20260808 11:48:23.835870 6235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:23.835168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:44759)
0808 11:48:23.835624 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:23.835704 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260808 11:48:24.176103 6236 raft_consensus.cc:482] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:24.176371 6236 raft_consensus.cc:504] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:48:24.177021 6236 leader_election.cc:288] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
W20260808 11:48:24.178087 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
W20260808 11:48:24.178351 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
I20260808 11:48:24.178496 5668 leader_election.cc:302] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f43ff67b9a9a4f56ab9f8fce8e7b4c65; no voters: 2b36bdbb96cd4018a168b925706c178f, 6a07ea4184fa421f976f80be31a78f53
I20260808 11:48:24.178838 6236 raft_consensus.cc:2726] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:48:24.305559 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:24.304771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:38961)
0808 11:48:24.305244 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:24.305383 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":246,"threads_started":1}
I20260808 11:48:24.385319 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:24.384531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:42325)
0808 11:48:24.385008 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:24.385123 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":278,"threads_started":1}
I20260808 11:48:24.837626 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:24.836844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:34823)
0808 11:48:24.837363 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:24.837446 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":299,"threads_started":1}
I20260808 11:48:25.307212 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:25.306368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:56221)
0808 11:48:25.306929 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:25.307036 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":274,"threads_started":1}
I20260808 11:48:25.387212 6242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:25.386492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:57809)
0808 11:48:25.386964 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:25.387047 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":282,"threads_started":1}
I20260808 11:48:25.839459 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:25.838689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:46887)
0808 11:48:25.839204 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:25.839288 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":285,"threads_started":1}
I20260808 11:48:26.308885 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:26.308080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:47287)
0808 11:48:26.308617 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:26.308708 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":265,"threads_started":1}
I20260808 11:48:26.388867 6248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:26.388091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:56645)
0808 11:48:26.388585 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:26.388681 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":246,"threads_started":1}
I20260808 11:48:26.841022 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:26.840310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:52417)
0808 11:48:26.840761 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:26.840844 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":249,"threads_started":1}
I20260808 11:48:27.154776 6250 raft_consensus.cc:482] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:27.155000 6250 raft_consensus.cc:504] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:48:27.155608 6250 leader_election.cc:288] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
W20260808 11:48:27.156626 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
W20260808 11:48:27.156829 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
I20260808 11:48:27.156917 5668 leader_election.cc:302] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f43ff67b9a9a4f56ab9f8fce8e7b4c65; no voters: 2b36bdbb96cd4018a168b925706c178f, 6a07ea4184fa421f976f80be31a78f53
I20260808 11:48:27.157156 6250 raft_consensus.cc:2726] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:48:27.310827 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:27.310062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:47425)
0808 11:48:27.310501 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:27.310584 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":240,"threads_started":1}
I20260808 11:48:27.390492 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:27.389753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:50247)
0808 11:48:27.390195 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:27.390290 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":256,"threads_started":1}
I20260808 11:48:27.842937 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:27.841953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:47575)
0808 11:48:27.842577 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:27.842675 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":327,"threads_started":1}
I20260808 11:48:28.312490 6256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:28.311687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:53481)
0808 11:48:28.312200 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:28.312293 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":232,"threads_started":1}
I20260808 11:48:28.391957 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:28.391301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:58609)
0808 11:48:28.391687 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:28.391780 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":206,"threads_started":1}
I20260808 11:48:28.844521 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:28.843829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:49177)
0808 11:48:28.844261 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:28.844350 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":234,"threads_started":1}
I20260808 11:48:29.314195 6260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:29.313439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:42157)
0808 11:48:29.313895 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:29.313984 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":236,"threads_started":1}
I20260808 11:48:29.393720 6261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:29.392971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:52327)
0808 11:48:29.393464 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:29.393548 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":266,"threads_started":1}
I20260808 11:48:29.846407 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:29.845620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:60857)
0808 11:48:29.846127 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:29.846232 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":257,"threads_started":1}
I20260808 11:48:30.315820 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:30.315117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:43227)
0808 11:48:30.315580 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:30.315667 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":230,"threads_started":1}
I20260808 11:48:30.395591 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:30.394875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:47229)
0808 11:48:30.395314 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:30.395399 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":221,"threads_started":1}
I20260808 11:48:30.814479 6266 raft_consensus.cc:482] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:30.814772 6266 raft_consensus.cc:504] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:48:30.815450 6266 leader_election.cc:288] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
W20260808 11:48:30.816401 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
W20260808 11:48:30.816637 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
I20260808 11:48:30.816744 5668 leader_election.cc:302] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f43ff67b9a9a4f56ab9f8fce8e7b4c65; no voters: 2b36bdbb96cd4018a168b925706c178f, 6a07ea4184fa421f976f80be31a78f53
I20260808 11:48:30.816996 6266 raft_consensus.cc:2726] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:48:30.847975 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:30.847297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:52121)
0808 11:48:30.847737 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:30.847812 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":246,"threads_started":1}
I20260808 11:48:31.317463 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:31.316667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:39099)
0808 11:48:31.317207 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:31.317289 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":334,"threads_started":1}
I20260808 11:48:31.397164 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:31.396473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:36199)
0808 11:48:31.396905 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:31.396984 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":242,"threads_started":1}
I20260808 11:48:31.849570 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:31.848813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:43351)
0808 11:48:31.849321 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:31.849402 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":269,"threads_started":1}
I20260808 11:48:32.319396 6274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:32.318344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:60131)
0808 11:48:32.319061 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:32.319183 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":403,"threads_started":1}
I20260808 11:48:32.398785 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:32.398000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:41421)
0808 11:48:32.398481 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:32.398559 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":277,"threads_started":1}
I20260808 11:48:32.851203 6279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:32.850467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:59253)
0808 11:48:32.850956 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:32.851031 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":292,"threads_started":1}
I20260808 11:48:33.321385 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:33.320559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:56123)
0808 11:48:33.321127 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:33.321217 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":330,"threads_started":1}
I20260808 11:48:33.400516 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:33.399693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:34825)
0808 11:48:33.400206 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:33.400305 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":261,"threads_started":1}
I20260808 11:48:33.852926 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:33.852164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:41527)
0808 11:48:33.852672 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:33.852767 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":267,"threads_started":1}
I20260808 11:48:34.323474 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:34.322584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:36165)
0808 11:48:34.323128 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:34.323277 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":313,"threads_started":1}
I20260808 11:48:34.402235 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:34.401521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:42719)
0808 11:48:34.401954 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:34.402034 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":245,"threads_started":1}
I20260808 11:48:34.854851 6289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:34.854032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:52169)
0808 11:48:34.854510 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:34.854593 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":262,"threads_started":1}
I20260808 11:48:35.325120 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:35.324353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:57473)
0808 11:48:35.324799 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:35.324895 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":247,"threads_started":1}
I20260808 11:48:35.404088 6291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:35.403391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:34621)
0808 11:48:35.403828 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:35.403916 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":233,"threads_started":1}
I20260808 11:48:35.801921 6292 raft_consensus.cc:482] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:35.802150 6292 raft_consensus.cc:504] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b36bdbb96cd4018a168b925706c178f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 46095 } } peers { permanent_uuid: "f43ff67b9a9a4f56ab9f8fce8e7b4c65" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 42487 } } peers { permanent_uuid: "6a07ea4184fa421f976f80be31a78f53" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37611 } }
I20260808 11:48:35.803032 6292 leader_election.cc:288] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095), 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611)
W20260808 11:48:35.804100 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b36bdbb96cd4018a168b925706c178f (127.13.158.66:46095): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
W20260808 11:48:35.804286 5668 leader_election.cc:341] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a07ea4184fa421f976f80be31a78f53 (127.13.158.67:37611): Not found: Tablet not found: 6aaeb6dcbbbd45bbb38ae942b40827c0
I20260808 11:48:35.804380 5668 leader_election.cc:302] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f43ff67b9a9a4f56ab9f8fce8e7b4c65; no voters: 2b36bdbb96cd4018a168b925706c178f, 6a07ea4184fa421f976f80be31a78f53
I20260808 11:48:35.804600 6292 raft_consensus.cc:2726] T 6aaeb6dcbbbd45bbb38ae942b40827c0 P f43ff67b9a9a4f56ab9f8fce8e7b4c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:48:35.856585 6296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:35.855909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.66:48527)
0808 11:48:35.856322 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:35.856436 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":222,"threads_started":1}
I20260808 11:48:36.299652 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 5586
I20260808 11:48:36.319566 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:36.300723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.13.158.65:42487)
0808 11:48:36.301371 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:36.301382 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:36.301415 (+ 33us) server_negotiation.cc:407] Connection header received
0808 11:48:36.301517 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:36.301529 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:36.301863 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:36.302067 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:36.303678 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:36.304687 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:36.305880 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:36.306440 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:36.306539 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:36.314287 (+ 7748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:36.314298 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:36.314312 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:36.314344 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:36.317098 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:36.317736 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:36.317743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:36.317745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:36.317869 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:36.318178 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:36.318181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:36.318183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:36.318527 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:36.318871 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:36.319126 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 11:48:36.319317 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":285,"threads_started":1}
I20260808 11:48:36.320853 5828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:36.326908 6299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:36.326085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.67:51985)
0808 11:48:36.326572 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:36.326686 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":270,"threads_started":1}
I20260808 11:48:36.410455 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:36.408903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:33657 (local address 127.13.158.65:34545)
0808 11:48:36.410086 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:36.410226 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.126:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1055,"thread_start_us":900,"threads_started":1}
I20260808 11:48:36.410941 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 5586
I20260808 11:48:36.441474 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 5840
I20260808 11:48:36.454512 6301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:36.442218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35408 (local address 127.13.158.66:46095)
0808 11:48:36.442830 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:36.442835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:36.442851 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:48:36.442905 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:36.442909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:36.442966 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:36.443053 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:36.444482 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:36.445219 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:36.446385 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:36.446846 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:36.446940 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:36.449479 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:36.449484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:36.449487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:36.449518 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:36.452215 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:36.452791 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:36.452794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:36.452797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:36.452918 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:36.453240 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:36.453243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:36.453245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:36.453484 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:36.453664 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:36.453941 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 11:48:36.454105 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":349,"threads_started":1}
I20260808 11:48:36.455251 6020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:36.537220 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 5840
I20260808 11:48:36.564388 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 6030
I20260808 11:48:36.578471 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:36.565288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.13.158.67:37611)
0808 11:48:36.565871 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:36.565876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:36.565891 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:48:36.565946 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:36.565950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:36.566007 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:36.566174 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:36.567716 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:36.568468 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:36.569781 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:36.570227 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:36.570365 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:36.573101 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:36.573106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:36.573110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:36.573142 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:36.575833 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:36.576475 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:36.576481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:36.576485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:36.576629 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:36.577024 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:36.577029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:36.577032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:36.577291 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:36.577579 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:36.577860 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 11:48:36.578086 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":312,"threads_started":1}
I20260808 11:48:36.579234 6205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:36.664390 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 6030
2026-08-08T11:48:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64829 ms)
[----------] 30 tests from SecurityITest (302159 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:48:36 dist-test-slave-39l8 krb5kdc[6309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:48:36 dist-test-slave-39l8 krb5kdc[6309](info): set up 2 sockets
Aug 08 11:48:36 dist-test-slave-39l8 krb5kdc[6309](info): commencing operation
krb5kdc: starting...
W20260808 11:48:37.956840 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:48:38 dist-test-slave-39l8 krb5kdc[6309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:48:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:48:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:48:38.275070 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43187
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:43187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:38.619473 6328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:38.619756 6328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:38.619921 6328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:38.627075 6328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:48:38.627199 6328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:38.627288 6328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:38.627393 6328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:48:38.627455 6328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:48:38.637166 6328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:43187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:38.638526 6328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:38.639832 6328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:38.652513 6336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:38.652611 6341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:38.652654 6338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:38.660387 6328 server_base.cc:1034] running on GCE node
I20260808 11:48:38.661562 6328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:38.663609 6328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:38.664832 6328 hybrid_clock.cc:648] HybridClock initialized: now 1786189718664776 us; error 56 us; skew 500 ppm
I20260808 11:48:38.665221 6328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:38 dist-test-slave-39l8 krb5kdc[6309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189718, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:38.668409 6328 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:48:38.669930 6328 webserver.cc:460] Webserver started at http://127.13.158.126:44483/ using document root <none> and password file <none>
I20260808 11:48:38.670599 6328 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:38.670774 6328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:38.671056 6328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:38.674041 6328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "98bad6bf479643119c0c59f8551bfe7e"
format_stamp: "Formatted at 2026-08-08 11:48:38 on dist-test-slave-39l8"
server_key: "ec84db106b6e145bb32b53533140cad5"
server_key_iv: "aff8982f77038412f2b3b8b082ddea53"
server_key_version: "encryptionkey@0"
I20260808 11:48:38.674805 6328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "98bad6bf479643119c0c59f8551bfe7e"
format_stamp: "Formatted at 2026-08-08 11:48:38 on dist-test-slave-39l8"
server_key: "ec84db106b6e145bb32b53533140cad5"
server_key_iv: "aff8982f77038412f2b3b8b082ddea53"
server_key_version: "encryptionkey@0"
I20260808 11:48:38.679886 6328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 11:48:38.683347 6352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:38.684639 6328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:38.684834 6328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "98bad6bf479643119c0c59f8551bfe7e"
format_stamp: "Formatted at 2026-08-08 11:48:38 on dist-test-slave-39l8"
server_key: "ec84db106b6e145bb32b53533140cad5"
server_key_iv: "aff8982f77038412f2b3b8b082ddea53"
server_key_version: "encryptionkey@0"
I20260808 11:48:38.685029 6328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:38.707718 6328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:38.711609 6328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:38.711894 6328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:38.730623 6328 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:43187
I20260808 11:48:38.730647 6429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:43187 every 8 connection(s)
I20260808 11:48:38.732023 6328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:48:38.734417 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 6328
I20260808 11:48:38.734666 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:48:38.735342 13945 external_mini_cluster.cc:1428] Setting key c6aef13a41443e71990179791b6ae0ff
I20260808 11:48:38.735985 6430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:48:38 dist-test-slave-39l8 krb5kdc[6309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:38.750836 6430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e: Bootstrap starting.
I20260808 11:48:38.754153 6430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:38.755224 6430 log.cc:824] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:38.757984 6430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e: No bootstrap required, opened a new log
I20260808 11:48:38.758303 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:38.737380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.13.158.126:43187)
0808 11:48:38.738175 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:38.738186 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:38.738225 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:48:38.739778 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:38.739799 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:38.740151 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:38.740465 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:38.742272 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:38.743573 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:38.745536 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:38.746146 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:38.746269 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:38.751224 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:38.751231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:38.751243 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:38.751282 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:38.754213 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:38.755002 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:38.755010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:38.755016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:38.755175 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:38.755622 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:38.755628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:38.755631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:38.756158 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:38.756447 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:38.756906 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 11:48:38.757356 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":373,"threads_started":1}
I20260808 11:48:38.765923 6430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bad6bf479643119c0c59f8551bfe7e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43187 } }
I20260808 11:48:38.766273 6430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:38.766377 6430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bad6bf479643119c0c59f8551bfe7e, State: Initialized, Role: FOLLOWER
I20260808 11:48:38.766950 6430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bad6bf479643119c0c59f8551bfe7e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43187 } }
I20260808 11:48:38.767189 6430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:48:38.767349 6430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:48:38.767539 6430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:38.769546 6430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bad6bf479643119c0c59f8551bfe7e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43187 } }
I20260808 11:48:38.770051 6430 leader_election.cc:302] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98bad6bf479643119c0c59f8551bfe7e; no voters:
I20260808 11:48:38.770620 6430 leader_election.cc:288] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:48:38.770797 6435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:38.771481 6435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [term 1 LEADER]: Becoming Leader. State: Replica: 98bad6bf479643119c0c59f8551bfe7e, State: Running, Role: LEADER
I20260808 11:48:38.772033 6435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bad6bf479643119c0c59f8551bfe7e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43187 } }
I20260808 11:48:38.772696 6430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:48:38.774688 6436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98bad6bf479643119c0c59f8551bfe7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bad6bf479643119c0c59f8551bfe7e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43187 } } }
I20260808 11:48:38.774881 6438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98bad6bf479643119c0c59f8551bfe7e. Latest consensus state: current_term: 1 leader_uuid: "98bad6bf479643119c0c59f8551bfe7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bad6bf479643119c0c59f8551bfe7e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43187 } } }
I20260808 11:48:38.775125 6436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:38.775161 6438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:38.776726 6446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:48:38.781543 6446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:48:38.789662 6446 catalog_manager.cc:1347] Generated new cluster ID: 52df5327d9924909b73d59d07b88bbaf
I20260808 11:48:38.789773 6446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:48:38.803200 6446 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:48:38.804447 6446 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:48:38.812733 6446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e: Generated new TSK 0
I20260808 11:48:38.813475 6446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:48:38.888480 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43187
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:39.249464 6466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:39.249760 6466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:39.249940 6466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:39.257296 6466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:39.257463 6466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:39.257647 6466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:48:39.267741 6466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:39.269089 6466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:39.270421 6466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:39.284078 6477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:39.284226 6473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:39.284147 6475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:39.285305 6466 server_base.cc:1034] running on GCE node
I20260808 11:48:39.285924 6466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:39.286938 6466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:39.288173 6466 hybrid_clock.cc:648] HybridClock initialized: now 1786189719288089 us; error 57 us; skew 500 ppm
I20260808 11:48:39.288574 6466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:39 dist-test-slave-39l8 krb5kdc[6309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189719, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:39.291924 6466 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:48:39.293509 6466 webserver.cc:460] Webserver started at http://127.13.158.65:35763/ using document root <none> and password file <none>
I20260808 11:48:39.294245 6466 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:39.294385 6466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:39.294772 6466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:39.297713 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76"
format_stamp: "Formatted at 2026-08-08 11:48:39 on dist-test-slave-39l8"
server_key: "9a1aa6eaae36f8ec2d9ed90c7392650b"
server_key_iv: "4efdf87bc738340b1798bc516768776f"
server_key_version: "encryptionkey@0"
I20260808 11:48:39.298497 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76"
format_stamp: "Formatted at 2026-08-08 11:48:39 on dist-test-slave-39l8"
server_key: "9a1aa6eaae36f8ec2d9ed90c7392650b"
server_key_iv: "4efdf87bc738340b1798bc516768776f"
server_key_version: "encryptionkey@0"
I20260808 11:48:39.303949 6466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 11:48:39.307474 6487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:39.308758 6466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:39.308939 6466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76"
format_stamp: "Formatted at 2026-08-08 11:48:39 on dist-test-slave-39l8"
server_key: "9a1aa6eaae36f8ec2d9ed90c7392650b"
server_key_iv: "4efdf87bc738340b1798bc516768776f"
server_key_version: "encryptionkey@0"
I20260808 11:48:39.309159 6466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:39.325044 6466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:39.328913 6466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:39.329214 6466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:39.330170 6466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:39.331523 6466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:39.331632 6466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:39.331776 6466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:39.331861 6466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:39.364646 6466 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:39009
I20260808 11:48:39.364686 6645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:39009 every 8 connection(s)
I20260808 11:48:39.365926 6466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:48:39.370633 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 6466
I20260808 11:48:39.370877 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:48:39.371536 13945 external_mini_cluster.cc:1428] Setting key b0308cc0841cd2c607b4f32659b84f21
Aug 08 11:48:39 dist-test-slave-39l8 krb5kdc[6309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189719, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:39.385598 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:39.368446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:47523 (local address 127.13.158.126:43187)
0808 11:48:39.368912 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:39.368916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:39.369842 (+ 926us) server_negotiation.cc:407] Connection header received
0808 11:48:39.371857 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:39.371863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:39.371961 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:39.372092 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:39.373858 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:39.374614 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:39.376185 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:39.376581 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:39.376720 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:39.380158 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:39.380163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:39.380166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:39.380196 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:39.382592 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:39.383310 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:39.383317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:39.383325 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:39.383443 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:39.383914 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:39.383919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:39.383922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:39.384183 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:39.384338 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:39.385003 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 11:48:39.385253 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":234,"threads_started":1}
I20260808 11:48:39.386837 6652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:39.368957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43187 (local address 127.13.158.65:47523)
0808 11:48:39.369677 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:39.369727 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:39.371562 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:39.372337 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:39.372345 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:39.373080 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:39.373698 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:39.373717 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:39.374802 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:39.374806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:39.376037 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:39.376048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:39.376808 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:39.376817 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:39.377042 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:39.377761 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:39.377782 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:39.379944 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:39.382750 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:39.382760 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:39.382775 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:39.383168 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:39.383553 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:39.383558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:39.383561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:39.383791 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:39.384313 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:39.384321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:39.384632 (+ 311us) client_negotiation.cc:769] Sending connection context
0808 11:48:39.384887 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 11:48:39.385407 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":275,"threads_started":1}
I20260808 11:48:39.388518 6646 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43187
I20260808 11:48:39.388849 6646 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:39.389722 6646 heartbeater.cc:511] Master 127.13.158.126:43187 requested a full tablet report, sending...
I20260808 11:48:39.391844 6378 ts_manager.cc:194] Registered new tserver with Master: 4f3b1e15483d4ba7b7b3b09131ab3d76 (127.13.158.65:39009)
I20260808 11:48:39.393260 6378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:47523
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:48:39.437397 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43187
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:39.776046 6657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:39.776391 6657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:39.776566 6657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:39.783668 6657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:39.783811 6657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:39.783985 6657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:48:39.793830 6657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:39.795217 6657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:39.796581 6657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:39.809669 6671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:39.809834 6675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:39.809849 6670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:39.810662 6657 server_base.cc:1034] running on GCE node
I20260808 11:48:39.811324 6657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:39.812182 6657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:39.813411 6657 hybrid_clock.cc:648] HybridClock initialized: now 1786189719813374 us; error 51 us; skew 500 ppm
I20260808 11:48:39.813759 6657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:39 dist-test-slave-39l8 krb5kdc[6309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189719, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:39.817126 6657 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:48:39.818511 6657 webserver.cc:460] Webserver started at http://127.13.158.66:40341/ using document root <none> and password file <none>
I20260808 11:48:39.819231 6657 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:39.819406 6657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:39.819720 6657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:39.822521 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "4985e16ecc4047f7be7c254de9288e0f"
format_stamp: "Formatted at 2026-08-08 11:48:39 on dist-test-slave-39l8"
server_key: "0d91ee4c6ed277a3c154f495c2785d3b"
server_key_iv: "84d7e47a48a401caf7c2b248af984a01"
server_key_version: "encryptionkey@0"
I20260808 11:48:39.823308 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "4985e16ecc4047f7be7c254de9288e0f"
format_stamp: "Formatted at 2026-08-08 11:48:39 on dist-test-slave-39l8"
server_key: "0d91ee4c6ed277a3c154f495c2785d3b"
server_key_iv: "84d7e47a48a401caf7c2b248af984a01"
server_key_version: "encryptionkey@0"
I20260808 11:48:39.828325 6657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:48:39.831604 6687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:39.832811 6657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:39.832984 6657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "4985e16ecc4047f7be7c254de9288e0f"
format_stamp: "Formatted at 2026-08-08 11:48:39 on dist-test-slave-39l8"
server_key: "0d91ee4c6ed277a3c154f495c2785d3b"
server_key_iv: "84d7e47a48a401caf7c2b248af984a01"
server_key_version: "encryptionkey@0"
I20260808 11:48:39.833177 6657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:39.851712 6657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:39.855436 6657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:39.855737 6657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:39.856685 6657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:39.857937 6657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:39.858038 6657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:39.858182 6657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:39.858260 6657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:39.893111 6657 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:39905
I20260808 11:48:39.893169 6852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:39905 every 8 connection(s)
I20260808 11:48:39.894515 6657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:48:39.897997 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 6657
I20260808 11:48:39.898214 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:48:39.898830 13945 external_mini_cluster.cc:1428] Setting key 27bbc46644f85d89eb7edebfe8527711
Aug 08 11:48:39 dist-test-slave-39l8 krb5kdc[6309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189719, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:39.918390 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:39.897057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:53277 (local address 127.13.158.126:43187)
0808 11:48:39.897544 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:39.897550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:39.898693 (+ 1143us) server_negotiation.cc:407] Connection header received
0808 11:48:39.900931 (+ 2238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:39.900936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:39.901031 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:39.901142 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:39.903327 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:39.904057 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:39.906483 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:39.906957 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:39.907050 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:39.911538 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:39.911543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:39.911546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:39.911578 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:39.914871 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:39.915670 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:39.915677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:39.915682 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:39.915865 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:39.916326 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:39.916334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:39.916339 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:39.916764 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:39.916949 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:39.917616 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 11:48:39.917884 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":237,"threads_started":1}
I20260808 11:48:39.918998 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:39.897612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43187 (local address 127.13.158.66:53277)
0808 11:48:39.898503 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:39.898556 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:39.900685 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:39.901447 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:39.901459 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:39.902278 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:39.903151 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:39.903181 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:39.904231 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:39.904239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:39.906339 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:39.906356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:39.907211 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:39.907221 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:39.907551 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:39.908292 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:39.908333 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:39.911271 (+ 2938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:39.915060 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:39.915069 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:39.915094 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:39.915507 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:39.916000 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:39.916003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:39.916005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:39.916196 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:39.916918 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:39.916923 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:39.917240 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 11:48:39.917464 (+ 224us) client_negotiation.cc:240] Negotiation successful
0808 11:48:39.917864 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":305,"threads_started":1}
I20260808 11:48:39.920629 6854 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43187
I20260808 11:48:39.920961 6854 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:39.921847 6854 heartbeater.cc:511] Master 127.13.158.126:43187 requested a full tablet report, sending...
I20260808 11:48:39.923565 6378 ts_manager.cc:194] Registered new tserver with Master: 4985e16ecc4047f7be7c254de9288e0f (127.13.158.66:39905)
I20260808 11:48:39.924422 6378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:53277
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:48:39.970192 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43187
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:40255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:40.316543 6861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:40.316845 6861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:40.317054 6861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:40.324537 6861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:40.324680 6861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:40.324873 6861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:48:40.334929 6861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:40255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:40.336331 6861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:40.337826 6861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:40.351773 6866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:40.351742 6869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:40.351675 6867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:40.352684 6861 server_base.cc:1034] running on GCE node
I20260808 11:48:40.353382 6861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:40.354368 6861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:40.355602 6861 hybrid_clock.cc:648] HybridClock initialized: now 1786189720355555 us; error 34 us; skew 500 ppm
I20260808 11:48:40.355938 6861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:40 dist-test-slave-39l8 krb5kdc[6309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189720, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:40.359184 6861 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:48:40.360769 6861 webserver.cc:460] Webserver started at http://127.13.158.67:46147/ using document root <none> and password file <none>
I20260808 11:48:40.361545 6861 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:40.361708 6861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:40.362064 6861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:40.365229 6861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "527e33a1f23a45118e9d54d5f3b315dc"
format_stamp: "Formatted at 2026-08-08 11:48:40 on dist-test-slave-39l8"
server_key: "80a26d782fa6aaf5645db757ac72aa70"
server_key_iv: "155a3b418014808c2393ea37643c5e4f"
server_key_version: "encryptionkey@0"
I20260808 11:48:40.366029 6861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "527e33a1f23a45118e9d54d5f3b315dc"
format_stamp: "Formatted at 2026-08-08 11:48:40 on dist-test-slave-39l8"
server_key: "80a26d782fa6aaf5645db757ac72aa70"
server_key_iv: "155a3b418014808c2393ea37643c5e4f"
server_key_version: "encryptionkey@0"
I20260808 11:48:40.371575 6861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:48:40.375368 6876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:40.376667 6861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:40.376850 6861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "527e33a1f23a45118e9d54d5f3b315dc"
format_stamp: "Formatted at 2026-08-08 11:48:40 on dist-test-slave-39l8"
server_key: "80a26d782fa6aaf5645db757ac72aa70"
server_key_iv: "155a3b418014808c2393ea37643c5e4f"
server_key_version: "encryptionkey@0"
I20260808 11:48:40.377058 6861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:40.396106 6646 heartbeater.cc:503] Master 127.13.158.126:43187 was elected leader, sending a full tablet report...
I20260808 11:48:40.408776 6861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:40.412925 6861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:40.413295 6861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:40.414448 6861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:40.415920 6861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:40.415995 6861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:40.416152 6861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:40.416204 6861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:40.451524 6861 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:38429
I20260808 11:48:40.451553 7040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:38429 every 8 connection(s)
I20260808 11:48:40.452931 6861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:48:40.462025 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 6861
I20260808 11:48:40.462250 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:48:40.462849 13945 external_mini_cluster.cc:1428] Setting key aa884752058c80df4e779d7d8658805a
Aug 08 11:48:40 dist-test-slave-39l8 krb5kdc[6309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189720, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:40.473135 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.455660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:54125 (local address 127.13.158.126:43187)
0808 11:48:40.456175 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.456181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.457169 (+ 988us) server_negotiation.cc:407] Connection header received
0808 11:48:40.459056 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.459060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.459148 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.459251 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:40.460720 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.461503 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.463838 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.464258 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.464408 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.467840 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:40.467844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:40.467848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:40.467879 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:40.470337 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:40.471076 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:40.471080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:40.471083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:40.471206 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:40.471588 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:40.471591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:40.471593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:40.471858 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:40.472045 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.472701 (+ 656us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.472868 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":240,"threads_started":1}
I20260808 11:48:40.474140 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.456302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43187 (local address 127.13.158.67:54125)
0808 11:48:40.456998 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:40.457044 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:40.458658 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:40.459458 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:40.459466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:40.460001 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:40.460562 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.460578 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.461682 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.461689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.463678 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.463695 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.464506 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.464515 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.464757 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.465457 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:40.465480 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:40.467664 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:40.470461 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:40.470467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:40.470480 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:40.470964 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:40.471300 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:40.471303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:40.471305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:40.471488 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:40.471963 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:40.471969 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:40.472289 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 11:48:40.472587 (+ 298us) client_negotiation.cc:240] Negotiation successful
0808 11:48:40.472962 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":284,"threads_started":1}
I20260808 11:48:40.475570 7041 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43187
I20260808 11:48:40.475872 7041 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:40.476701 7041 heartbeater.cc:511] Master 127.13.158.126:43187 requested a full tablet report, sending...
I20260808 11:48:40.478403 6378 ts_manager.cc:194] Registered new tserver with Master: 527e33a1f23a45118e9d54d5f3b315dc (127.13.158.67:38429)
I20260808 11:48:40.479219 6378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:54125
I20260808 11:48:40.490549 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:48:40.506935 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.495070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.13.158.126:43187)
0808 11:48:40.495293 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.495303 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.495754 (+ 451us) server_negotiation.cc:407] Connection header received
0808 11:48:40.495892 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.495896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.495970 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.496050 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:40.497504 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.498210 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.499472 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.499848 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.499938 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.502153 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:40.502158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:40.502161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:40.502191 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:40.504650 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:40.505218 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:40.505222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:40.505224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:40.505336 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:40.505651 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:40.505654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:40.505656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:40.505943 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:40.506113 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.506413 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.506571 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:48:40.510852 6378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:48:40.517918 6378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:48:40.544085 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.529698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39009 (local address 127.0.0.1:36884)
0808 11:48:40.530518 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:40.530579 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:40.530829 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:40.532425 (+ 1596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:40.532437 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:40.532479 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:40.532994 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.533011 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.535910 (+ 2899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.535919 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.539383 (+ 3464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.539405 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.543009 (+ 3604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.543019 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.543271 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.543336 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 11:48:40.543459 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 11:48:40.543665 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":321,"threads_started":1}
I20260808 11:48:40.544457 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.530099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.13.158.65:39009)
0808 11:48:40.531626 (+ 1527us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.531639 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.531675 (+ 36us) server_negotiation.cc:407] Connection header received
0808 11:48:40.531808 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.531819 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.532133 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.533250 (+ 1117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:40.533531 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.535711 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.539674 (+ 3963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.542215 (+ 2541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.543560 (+ 1345us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.543825 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.543899 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.544076 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1286,"thread_start_us":1481,"threads_started":1}
I20260808 11:48:40.552419 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.529809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58214 (local address 127.13.158.67:38429)
0808 11:48:40.542967 (+ 13158us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.542979 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.544589 (+ 1610us) server_negotiation.cc:407] Connection header received
0808 11:48:40.544931 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.544942 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.545255 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.545477 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:40.546334 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.547944 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.549842 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.551272 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.551401 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.551611 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.551884 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.552069 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12965,"thread_start_us":12760,"threads_started":1}
I20260808 11:48:40.552418 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.529707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:38429 (local address 127.0.0.1:58214)
0808 11:48:40.544466 (+ 14759us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:40.544488 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:40.544705 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:40.545900 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:40.545906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:40.545936 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:40.546129 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.546143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.548136 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.548142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.549642 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.549658 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.551449 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.551456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.551663 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.551707 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 11:48:40.551820 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 11:48:40.551967 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14656,"mutex_wait_us":986,"thread_start_us":15580,"threads_started":1}
I20260808 11:48:40.552752 6562 tablet_service.cc:1467] Processing CreateTablet for tablet 88d29c03b5794cceb1978c0c1f1c6201 (DEFAULT_TABLE table=test-table [id=0ce12043fb9b43b4bfafd481c5f6e1e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:40.553508 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.531407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:39905 (local address 127.0.0.1:35626)
0808 11:48:40.545235 (+ 13828us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:40.545253 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:40.545459 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:40.546333 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:40.546337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:40.546361 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:40.546827 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.546839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.548629 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.548635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.550541 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.550554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.552581 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.552588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.552797 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.552834 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 11:48:40.552939 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 11:48:40.553119 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13709,"thread_start_us":16861,"threads_started":1}
I20260808 11:48:40.554306 6562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d29c03b5794cceb1978c0c1f1c6201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:40.554816 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.531122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.13.158.66:39905)
0808 11:48:40.543438 (+ 12316us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.543448 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.545350 (+ 1902us) server_negotiation.cc:407] Connection header received
0808 11:48:40.545650 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.545657 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.545888 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.546060 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:40.547029 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.548490 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.550733 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.552039 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.552206 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.552416 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.553730 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.554467 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12147,"thread_start_us":12035,"threads_started":1}
I20260808 11:48:40.556504 6940 tablet_service.cc:1467] Processing CreateTablet for tablet 88d29c03b5794cceb1978c0c1f1c6201 (DEFAULT_TABLE table=test-table [id=0ce12043fb9b43b4bfafd481c5f6e1e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:40.558139 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d29c03b5794cceb1978c0c1f1c6201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:40.558534 6754 tablet_service.cc:1467] Processing CreateTablet for tablet 88d29c03b5794cceb1978c0c1f1c6201 (DEFAULT_TABLE table=test-table [id=0ce12043fb9b43b4bfafd481c5f6e1e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:40.559993 6754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d29c03b5794cceb1978c0c1f1c6201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:40.566123 7067 tablet_bootstrap.cc:492] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Bootstrap starting.
I20260808 11:48:40.569576 7067 tablet_bootstrap.cc:654] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:40.570658 7067 log.cc:824] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:40.573448 7067 tablet_bootstrap.cc:492] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: No bootstrap required, opened a new log
I20260808 11:48:40.573927 7067 ts_tablet_manager.cc:1397] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260808 11:48:40.574119 7070 tablet_bootstrap.cc:492] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Bootstrap starting.
I20260808 11:48:40.575882 7071 tablet_bootstrap.cc:492] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Bootstrap starting.
I20260808 11:48:40.577699 7070 tablet_bootstrap.cc:654] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:40.578982 7070 log.cc:824] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:40.579136 7071 tablet_bootstrap.cc:654] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:40.580423 7071 log.cc:824] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:40.583735 7070 tablet_bootstrap.cc:492] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: No bootstrap required, opened a new log
I20260808 11:48:40.583774 7071 tablet_bootstrap.cc:492] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: No bootstrap required, opened a new log
I20260808 11:48:40.584017 7070 ts_tablet_manager.cc:1397] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20260808 11:48:40.584028 7071 ts_tablet_manager.cc:1397] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 11:48:40.585743 7067 raft_consensus.cc:348] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.586166 7067 raft_consensus.cc:374] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:40.586292 7067 raft_consensus.cc:729] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3b1e15483d4ba7b7b3b09131ab3d76, State: Initialized, Role: FOLLOWER
I20260808 11:48:40.586915 7067 consensus_queue.cc:260] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.588678 7067 ts_tablet_manager.cc:1428] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260808 11:48:40.593281 7071 raft_consensus.cc:348] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.593715 7071 raft_consensus.cc:374] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:40.593822 7071 raft_consensus.cc:729] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4985e16ecc4047f7be7c254de9288e0f, State: Initialized, Role: FOLLOWER
I20260808 11:48:40.594484 7071 consensus_queue.cc:260] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.594422 7070 raft_consensus.cc:348] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.594929 7070 raft_consensus.cc:374] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:40.595078 7070 raft_consensus.cc:729] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527e33a1f23a45118e9d54d5f3b315dc, State: Initialized, Role: FOLLOWER
I20260808 11:48:40.595753 6854 heartbeater.cc:503] Master 127.13.158.126:43187 was elected leader, sending a full tablet report...
I20260808 11:48:40.595726 7070 consensus_queue.cc:260] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.596375 7071 ts_tablet_manager.cc:1428] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 11:48:40.596907 7041 heartbeater.cc:503] Master 127.13.158.126:43187 was elected leader, sending a full tablet report...
I20260808 11:48:40.597441 7070 ts_tablet_manager.cc:1428] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260808 11:48:40.618266 6647 tablet.cc:2369] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:40.637755 7078 raft_consensus.cc:482] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:40.638254 7078 raft_consensus.cc:504] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.640782 7078 leader_election.cc:288] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f3b1e15483d4ba7b7b3b09131ab3d76 (127.13.158.65:39009), 4985e16ecc4047f7be7c254de9288e0f (127.13.158.66:39905)
W20260808 11:48:40.646874 6855 tablet.cc:2369] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:40.648561 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.641119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:34549 (local address 127.13.158.65:39009)
0808 11:48:40.641510 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.641516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.641539 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:48:40.641750 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.641757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.641886 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.642043 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:40.642981 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.643824 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.645936 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.647299 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.647541 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.647655 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.647957 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.648086 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260808 11:48:40.648561 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.641019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39009 (local address 127.13.158.67:34549)
0808 11:48:40.641411 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:40.641429 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:40.641544 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:40.642346 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:40.642351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:40.642388 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:40.642816 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.642829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.643973 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.643983 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.645724 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.645739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.647456 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.647463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.647782 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.647808 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 11:48:40.647917 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 11:48:40.648060 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260808 11:48:40.650022 6587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d29c03b5794cceb1978c0c1f1c6201" candidate_uuid: "527e33a1f23a45118e9d54d5f3b315dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" is_pre_election: true
I20260808 11:48:40.651101 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.641933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:35227 (local address 127.13.158.66:39905)
0808 11:48:40.642229 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.642238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.642874 (+ 636us) server_negotiation.cc:407] Connection header received
0808 11:48:40.643117 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.643122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.643181 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.643278 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:40.644314 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.645015 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.648177 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.649832 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.650063 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.650131 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.650479 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.650620 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:48:40.650929 6587 raft_consensus.cc:2454] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 527e33a1f23a45118e9d54d5f3b315dc in term 0.
I20260808 11:48:40.651228 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.641497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:39905 (local address 127.13.158.67:35227)
0808 11:48:40.642734 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:40.642773 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:40.642963 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:40.643292 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:40.643297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:40.643320 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:40.644115 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.644130 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.645207 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.645212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.648023 (+ 2811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:40.648036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.650127 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.650132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:40.650282 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.650306 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:48:40.650408 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 11:48:40.650555 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"thread_start_us":786,"threads_started":1}
I20260808 11:48:40.652263 6877 leader_election.cc:302] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3b1e15483d4ba7b7b3b09131ab3d76, 527e33a1f23a45118e9d54d5f3b315dc; no voters:
I20260808 11:48:40.653000 7078 raft_consensus.cc:2781] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:48:40.653374 7078 raft_consensus.cc:482] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:48:40.653527 7078 raft_consensus.cc:3037] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:40.653028 6779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d29c03b5794cceb1978c0c1f1c6201" candidate_uuid: "527e33a1f23a45118e9d54d5f3b315dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4985e16ecc4047f7be7c254de9288e0f" is_pre_election: true
I20260808 11:48:40.653937 6779 raft_consensus.cc:2454] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 527e33a1f23a45118e9d54d5f3b315dc in term 0.
I20260808 11:48:40.657343 7078 raft_consensus.cc:504] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.658218 7078 leader_election.cc:288] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [CANDIDATE]: Term 1 election: Requested vote from peers 4f3b1e15483d4ba7b7b3b09131ab3d76 (127.13.158.65:39009), 4985e16ecc4047f7be7c254de9288e0f (127.13.158.66:39905)
I20260808 11:48:40.658815 6587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d29c03b5794cceb1978c0c1f1c6201" candidate_uuid: "527e33a1f23a45118e9d54d5f3b315dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76"
I20260808 11:48:40.658910 6779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d29c03b5794cceb1978c0c1f1c6201" candidate_uuid: "527e33a1f23a45118e9d54d5f3b315dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4985e16ecc4047f7be7c254de9288e0f"
I20260808 11:48:40.659080 6587 raft_consensus.cc:3037] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:40.659272 6779 raft_consensus.cc:3037] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:40.662345 6587 raft_consensus.cc:2454] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 527e33a1f23a45118e9d54d5f3b315dc in term 1.
I20260808 11:48:40.662480 6779 raft_consensus.cc:2454] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 527e33a1f23a45118e9d54d5f3b315dc in term 1.
I20260808 11:48:40.662930 6877 leader_election.cc:302] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3b1e15483d4ba7b7b3b09131ab3d76, 527e33a1f23a45118e9d54d5f3b315dc; no voters:
I20260808 11:48:40.663331 7078 raft_consensus.cc:2781] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:40.664247 7078 raft_consensus.cc:686] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 1 LEADER]: Becoming Leader. State: Replica: 527e33a1f23a45118e9d54d5f3b315dc, State: Running, Role: LEADER
I20260808 11:48:40.664983 7078 consensus_queue.cc:237] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } }
I20260808 11:48:40.671973 6378 catalog_manager.cc:5495] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 527e33a1f23a45118e9d54d5f3b315dc (127.13.158.67). New cstate: current_term: 1 leader_uuid: "527e33a1f23a45118e9d54d5f3b315dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527e33a1f23a45118e9d54d5f3b315dc" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 } health_report { overall_health: UNKNOWN } } }
I20260808 11:48:40.698772 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.692013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.13.158.67:38429)
0808 11:48:40.692292 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.692300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.692396 (+ 96us) server_negotiation.cc:407] Connection header received
0808 11:48:40.692644 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.692648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.692751 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.692903 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:48:40.693468 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.694461 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.696124 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.696696 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.696836 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.697510 (+ 674us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:48:40.698005 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:48:40.698170 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.698315 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.698395 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260808 11:48:40.704910 7042 tablet.cc:2369] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:40.709838 6779 raft_consensus.cc:1261] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 1 FOLLOWER]: Refusing update from remote peer 527e33a1f23a45118e9d54d5f3b315dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:40.709848 6587 raft_consensus.cc:1261] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 1 FOLLOWER]: Refusing update from remote peer 527e33a1f23a45118e9d54d5f3b315dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:40.710911 7081 consensus_queue.cc:1035] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4985e16ecc4047f7be7c254de9288e0f" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:40.711479 7078 consensus_queue.cc:1035] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3b1e15483d4ba7b7b3b09131ab3d76" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:40.727739 7087 mvcc.cc:205] Tried to move back new op lower bound from 7316233096016867328 to 7316233095849029632. Current Snapshot: MvccSnapshot[applied={T|T < 7316233096016867328}]
I20260808 11:48:40.729981 7088 mvcc.cc:205] Tried to move back new op lower bound from 7316233096016867328 to 7316233095849029632. Current Snapshot: MvccSnapshot[applied={T|T < 7316233096016867328}]
I20260808 11:48:40.751623 6378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:40.752038 6378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:40.757534 6378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98bad6bf479643119c0c59f8551bfe7e: Sending DeleteTablet for 3 replicas of tablet 88d29c03b5794cceb1978c0c1f1c6201
I20260808 11:48:40.759006 6562 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d29c03b5794cceb1978c0c1f1c6201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:40 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:36884
I20260808 11:48:40.759025 6940 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d29c03b5794cceb1978c0c1f1c6201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:40 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:58214
I20260808 11:48:40.761505 6754 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d29c03b5794cceb1978c0c1f1c6201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:40 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:35626
I20260808 11:48:40.761936 7097 tablet_replica.cc:330] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: stopping tablet replica
I20260808 11:48:40.762562 7097 raft_consensus.cc:2229] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:40.762797 7098 tablet_replica.cc:330] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: stopping tablet replica
I20260808 11:48:40.763314 7098 raft_consensus.cc:2229] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:48:40.763398 7097 raft_consensus.cc:2258] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:40.763616 7099 tablet_replica.cc:330] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: stopping tablet replica
I20260808 11:48:40.764158 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 6466
I20260808 11:48:40.764146 7099 raft_consensus.cc:2229] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:40.764747 7098 raft_consensus.cc:2258] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:40.765172 7099 raft_consensus.cc:2258] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:40.767570 7097 ts_tablet_manager.cc:1905] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:40.769104 7099 ts_tablet_manager.cc:1905] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:40.771907 7098 ts_tablet_manager.cc:1905] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:40.778267 7099 ts_tablet_manager.cc:1918] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:40.778645 7099 log.cc:1196] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/88d29c03b5794cceb1978c0c1f1c6201
I20260808 11:48:40.779557 7097 ts_tablet_manager.cc:1918] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:40.779990 7097 log.cc:1196] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/88d29c03b5794cceb1978c0c1f1c6201
I20260808 11:48:40.780218 7099 ts_tablet_manager.cc:1939] T 88d29c03b5794cceb1978c0c1f1c6201 P 4985e16ecc4047f7be7c254de9288e0f: Deleting consensus metadata
I20260808 11:48:40.781241 7097 ts_tablet_manager.cc:1939] T 88d29c03b5794cceb1978c0c1f1c6201 P 4f3b1e15483d4ba7b7b3b09131ab3d76: Deleting consensus metadata
Aug 08 11:48:40 dist-test-slave-39l8 krb5kdc[6309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:48:40.783149 6353 catalog_manager.cc:4841] TS 4985e16ecc4047f7be7c254de9288e0f (127.13.158.66:39905): tablet 88d29c03b5794cceb1978c0c1f1c6201 (table test-table [id=0ce12043fb9b43b4bfafd481c5f6e1e6]) successfully deleted
I20260808 11:48:40.784317 6353 catalog_manager.cc:4841] TS 4f3b1e15483d4ba7b7b3b09131ab3d76 (127.13.158.65:39009): tablet 88d29c03b5794cceb1978c0c1f1c6201 (table test-table [id=0ce12043fb9b43b4bfafd481c5f6e1e6]) successfully deleted
I20260808 11:48:40.790343 7098 ts_tablet_manager.cc:1918] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:40.790655 7098 log.cc:1196] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/88d29c03b5794cceb1978c0c1f1c6201
I20260808 11:48:40.792032 7098 ts_tablet_manager.cc:1939] T 88d29c03b5794cceb1978c0c1f1c6201 P 527e33a1f23a45118e9d54d5f3b315dc: Deleting consensus metadata
I20260808 11:48:40.793512 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.765579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.13.158.65:39009)
0808 11:48:40.765796 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.765805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.767765 (+ 1960us) server_negotiation.cc:407] Connection header received
0808 11:48:40.768010 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.768017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.768104 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.768181 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:40.770117 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.771672 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.774334 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.775673 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.775789 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.784734 (+ 8945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:40.784746 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:40.784759 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:40.784820 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:40.789546 (+ 4726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:40.790425 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:40.790434 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:40.790438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:40.790762 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:40.791291 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:40.791297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:40.791300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:40.791808 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:40.792338 (+ 530us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.792599 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.792860 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:48:40.794456 6354 catalog_manager.cc:4841] TS 527e33a1f23a45118e9d54d5f3b315dc (127.13.158.67:38429): tablet 88d29c03b5794cceb1978c0c1f1c6201 (table test-table [id=0ce12043fb9b43b4bfafd481c5f6e1e6]) successfully deleted
I20260808 11:48:40.794801 6640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:40.890125 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 6466
I20260808 11:48:40.926388 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 6657
Aug 08 11:48:40 dist-test-slave-39l8 krb5kdc[6309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:48:40.945856 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:40.927515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.13.158.66:39905)
0808 11:48:40.927724 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:40.927730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:40.927750 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:48:40.927820 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:40.927825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:40.927907 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:40.928032 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:40.929763 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.931185 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.933141 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:40.933814 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:40.933951 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:40.939593 (+ 5642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:40.939605 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:40.939620 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:40.939654 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:40.942683 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:40.943580 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:40.943587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:40.943589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:40.943718 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:40.944126 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:40.944129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:40.944131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:40.944657 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:40.944876 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:40.945316 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 11:48:40.945483 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 11:48:40.946661 6848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:41.036980 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 6657
I20260808 11:48:41.061806 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 6861
Aug 08 11:48:41 dist-test-slave-39l8 krb5kdc[6309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:48:41.081185 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:41.062884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58228 (local address 127.13.158.67:38429)
0808 11:48:41.063078 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:41.063083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:41.063101 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:48:41.063160 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:41.063164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:41.063241 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:41.063328 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:41.065331 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:41.066150 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:41.068056 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:41.068487 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:41.068580 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:41.074531 (+ 5951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:41.074543 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:41.074558 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:41.074597 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:41.077994 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:41.078949 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:41.078956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:41.078959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:41.079110 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:41.079543 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:41.079546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:41.079548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:41.079917 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:41.080134 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:41.080676 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 11:48:41.080840 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 11:48:41.082188 7036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:41.202934 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 6861
I20260808 11:48:41.228384 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 6328
I20260808 11:48:41.229564 6425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:41.338436 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 6328
2026-08-08T11:48:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4674 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T11:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:48:41Z Disabled control of system clock
I20260808 11:48:41.440265 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43657
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:43657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:48:41.785705 7116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:41.786010 7116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:41.786142 7116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:41.793505 7116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:48:41.793654 7116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:41.793753 7116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:41.793903 7116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:48:41.793965 7116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:48:41.803184 7116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:43657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:41.804499 7116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:41.806371 7116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:41.818590 7124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:41.818625 7122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:41.818663 7128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:41.819425 7116 server_base.cc:1034] running on GCE node
I20260808 11:48:41.820210 7116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:41.821475 7116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:41.822760 7116 hybrid_clock.cc:648] HybridClock initialized: now 1786189721822678 us; error 45 us; skew 500 ppm
I20260808 11:48:41.823143 7116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:41.830135 7116 webserver.cc:460] Webserver started at http://127.13.158.126:44717/ using document root <none> and password file <none>
I20260808 11:48:41.830922 7116 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:41.831055 7116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:41.831389 7116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:41.834762 7116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "33fc3e32255a486babe0bfb8964787cf"
format_stamp: "Formatted at 2026-08-08 11:48:41 on dist-test-slave-39l8"
server_key: "cef50fec294ccabff60a460217681f97"
server_key_iv: "090d914e4aa26ff87253fdb7f8299449"
server_key_version: "encryptionkey@0"
I20260808 11:48:41.835594 7116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "33fc3e32255a486babe0bfb8964787cf"
format_stamp: "Formatted at 2026-08-08 11:48:41 on dist-test-slave-39l8"
server_key: "cef50fec294ccabff60a460217681f97"
server_key_iv: "090d914e4aa26ff87253fdb7f8299449"
server_key_version: "encryptionkey@0"
I20260808 11:48:41.840420 7116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:48:41.843858 7136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:41.845120 7116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:48:41.845314 7116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "33fc3e32255a486babe0bfb8964787cf"
format_stamp: "Formatted at 2026-08-08 11:48:41 on dist-test-slave-39l8"
server_key: "cef50fec294ccabff60a460217681f97"
server_key_iv: "090d914e4aa26ff87253fdb7f8299449"
server_key_version: "encryptionkey@0"
I20260808 11:48:41.845554 7116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:41.874221 7116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:41.875138 7116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:41.875461 7116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:41.894160 7116 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:43657
I20260808 11:48:41.894181 7201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:43657 every 8 connection(s)
I20260808 11:48:41.895548 7116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:48:41.899202 7202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:41.899804 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 7116
I20260808 11:48:41.900005 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:48:41.900450 13945 external_mini_cluster.cc:1428] Setting key e4df25c60366e095dc206c283d4235bd
I20260808 11:48:41.913564 7202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf: Bootstrap starting.
I20260808 11:48:41.915663 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:41.902400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.13.158.126:43657)
0808 11:48:41.903465 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:41.903480 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:41.903541 (+ 61us) server_negotiation.cc:407] Connection header received
0808 11:48:41.905907 (+ 2366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:41.905933 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:41.906494 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:41.906938 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:41.907441 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:41.909145 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:41.910502 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:41.911651 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:41.911825 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:41.912915 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:41.912925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:41.912942 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:41.913653 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:41.913750 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:41.913774 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:41.913883 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:41.914081 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 11:48:41.914398 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":357,"threads_started":1}
I20260808 11:48:41.917207 7202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:41.918262 7202 log.cc:824] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:41.920946 7202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf: No bootstrap required, opened a new log
I20260808 11:48:41.928320 7202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fc3e32255a486babe0bfb8964787cf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43657 } }
I20260808 11:48:41.928668 7202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:41.928784 7202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fc3e32255a486babe0bfb8964787cf, State: Initialized, Role: FOLLOWER
I20260808 11:48:41.929320 7202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fc3e32255a486babe0bfb8964787cf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43657 } }
I20260808 11:48:41.929560 7202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:48:41.929713 7202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:48:41.929868 7202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:41.932099 7202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fc3e32255a486babe0bfb8964787cf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43657 } }
I20260808 11:48:41.932646 7202 leader_election.cc:302] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33fc3e32255a486babe0bfb8964787cf; no voters:
I20260808 11:48:41.933194 7202 leader_election.cc:288] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:48:41.933333 7210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:41.934060 7210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [term 1 LEADER]: Becoming Leader. State: Replica: 33fc3e32255a486babe0bfb8964787cf, State: Running, Role: LEADER
I20260808 11:48:41.934674 7210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fc3e32255a486babe0bfb8964787cf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43657 } }
I20260808 11:48:41.935376 7202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:48:41.937445 7212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33fc3e32255a486babe0bfb8964787cf. Latest consensus state: current_term: 1 leader_uuid: "33fc3e32255a486babe0bfb8964787cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fc3e32255a486babe0bfb8964787cf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43657 } } }
I20260808 11:48:41.937386 7211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33fc3e32255a486babe0bfb8964787cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fc3e32255a486babe0bfb8964787cf" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43657 } } }
I20260808 11:48:41.937830 7211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:41.937829 7212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:41.939605 7222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:48:41.944630 7222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:48:41.953389 7222 catalog_manager.cc:1347] Generated new cluster ID: 0f1b210955aa4a2586e6be62ee550b28
I20260808 11:48:41.953495 7222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:48:41.969053 7222 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:48:41.970271 7222 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:48:41.981130 7222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf: Generated new TSK 0
I20260808 11:48:41.981990 7222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:48:41.987833 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43657
--builtin_ntp_servers=127.13.158.84:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:48:42.362505 7243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:42.362838 7243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:42.362993 7243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:42.369971 7243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:42.370121 7243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:42.370297 7243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:48:42.379060 7243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:42.380378 7243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:42.382154 7243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:42.395313 7254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:42.395355 7249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:42.395285 7251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:42.396428 7243 server_base.cc:1034] running on GCE node
I20260808 11:48:42.397025 7243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:42.397959 7243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:42.399194 7243 hybrid_clock.cc:648] HybridClock initialized: now 1786189722399135 us; error 46 us; skew 500 ppm
I20260808 11:48:42.399511 7243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:42.401959 7243 webserver.cc:460] Webserver started at http://127.13.158.65:36415/ using document root <none> and password file <none>
I20260808 11:48:42.402657 7243 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:42.402818 7243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:42.403142 7243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:42.406041 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "2b2e9ff52c664f16979d650938d7772c"
format_stamp: "Formatted at 2026-08-08 11:48:42 on dist-test-slave-39l8"
server_key: "ed6707e4c74ab7f062ef7dcdfbdfaaa2"
server_key_iv: "5e9117790de39697274993163b0604fd"
server_key_version: "encryptionkey@0"
I20260808 11:48:42.406939 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "2b2e9ff52c664f16979d650938d7772c"
format_stamp: "Formatted at 2026-08-08 11:48:42 on dist-test-slave-39l8"
server_key: "ed6707e4c74ab7f062ef7dcdfbdfaaa2"
server_key_iv: "5e9117790de39697274993163b0604fd"
server_key_version: "encryptionkey@0"
I20260808 11:48:42.411936 7243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 11:48:42.415663 7265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:42.417043 7243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:42.417213 7243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "2b2e9ff52c664f16979d650938d7772c"
format_stamp: "Formatted at 2026-08-08 11:48:42 on dist-test-slave-39l8"
server_key: "ed6707e4c74ab7f062ef7dcdfbdfaaa2"
server_key_iv: "5e9117790de39697274993163b0604fd"
server_key_version: "encryptionkey@0"
I20260808 11:48:42.417428 7243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:42.438865 7243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:42.439715 7243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:42.440030 7243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:42.440958 7243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:42.442406 7243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:42.442490 7243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:42.442559 7243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:42.442613 7243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:42.474977 7243 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:33019
I20260808 11:48:42.475070 7426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:33019 every 8 connection(s)
I20260808 11:48:42.476270 7243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:48:42.477571 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 7243
I20260808 11:48:42.477746 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:48:42.478250 13945 external_mini_cluster.cc:1428] Setting key c74d2dceed609dda48c557e7d1f58088
I20260808 11:48:42.481578 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43657
--builtin_ntp_servers=127.13.158.84:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:48:42.491297 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:42.478935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:48719 (local address 127.13.158.126:43657)
0808 11:48:42.479445 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:42.479451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:42.480980 (+ 1529us) server_negotiation.cc:407] Connection header received
0808 11:48:42.483731 (+ 2751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:42.483735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:42.483798 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:42.483888 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:42.485276 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:42.485990 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:42.488414 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:42.488816 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:42.488958 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:42.490379 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:42.490384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:42.490387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:42.490404 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:42.490443 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:42.490454 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:42.490558 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:42.490820 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 11:48:42.490918 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":265,"threads_started":1}
I20260808 11:48:42.492349 7430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:42.479504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43657 (local address 127.13.158.65:48719)
0808 11:48:42.480802 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:42.480861 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:42.483440 (+ 2579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:42.484152 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:42.484166 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:42.484274 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:42.485085 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:42.485113 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:42.486155 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:42.486163 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:42.488263 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:42.488280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:42.489045 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:42.489052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:42.489279 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:42.490007 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:42.490010 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:42.490074 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:42.490155 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:42.490559 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:42.490573 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:42.490583 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 11:48:42.490807 (+ 224us) client_negotiation.cc:240] Negotiation successful
0808 11:48:42.491213 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":343,"threads_started":1}
I20260808 11:48:42.493921 7427 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43657
I20260808 11:48:42.494231 7427 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:42.495165 7427 heartbeater.cc:511] Master 127.13.158.126:43657 requested a full tablet report, sending...
I20260808 11:48:42.497617 7157 ts_manager.cc:194] Registered new tserver with Master: 2b2e9ff52c664f16979d650938d7772c (127.13.158.65:33019)
I20260808 11:48:42.499873 7157 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.65:48719
W20260808 11:48:42.828649 7432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:42.828912 7432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:42.828999 7432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:42.836549 7432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:42.836697 7432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:42.836803 7432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:48:42.846100 7432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:42.847548 7432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:42.849421 7432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:42.862948 7438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:42.863000 7441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:42.862923 7439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:42.863811 7432 server_base.cc:1034] running on GCE node
I20260808 11:48:42.864411 7432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:42.865437 7432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:42.866688 7432 hybrid_clock.cc:648] HybridClock initialized: now 1786189722866640 us; error 63 us; skew 500 ppm
I20260808 11:48:42.867062 7432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:42.869316 7432 webserver.cc:460] Webserver started at http://127.13.158.66:34345/ using document root <none> and password file <none>
I20260808 11:48:42.870050 7432 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:42.870191 7432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:42.870518 7432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:42.873494 7432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "3973fc7eb7cc43e8bbbfc8fad1493224"
format_stamp: "Formatted at 2026-08-08 11:48:42 on dist-test-slave-39l8"
server_key: "a19cbff73718487185aced3e07e21701"
server_key_iv: "1a26cfea5a8a9c7692b3779f1ca14fcb"
server_key_version: "encryptionkey@0"
I20260808 11:48:42.874313 7432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "3973fc7eb7cc43e8bbbfc8fad1493224"
format_stamp: "Formatted at 2026-08-08 11:48:42 on dist-test-slave-39l8"
server_key: "a19cbff73718487185aced3e07e21701"
server_key_iv: "1a26cfea5a8a9c7692b3779f1ca14fcb"
server_key_version: "encryptionkey@0"
I20260808 11:48:42.879540 7432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 11:48:42.882949 7452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:42.884274 7432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:48:42.884454 7432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "3973fc7eb7cc43e8bbbfc8fad1493224"
format_stamp: "Formatted at 2026-08-08 11:48:42 on dist-test-slave-39l8"
server_key: "a19cbff73718487185aced3e07e21701"
server_key_iv: "1a26cfea5a8a9c7692b3779f1ca14fcb"
server_key_version: "encryptionkey@0"
I20260808 11:48:42.884680 7432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:42.914410 7432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:42.915457 7432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:42.915860 7432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:42.916807 7432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:42.918308 7432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:42.918392 7432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:42.918478 7432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:42.918536 7432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:42.953433 7432 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:44117
I20260808 11:48:42.953528 7613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:44117 every 8 connection(s)
I20260808 11:48:42.954770 7432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:48:42.956588 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 7432
I20260808 11:48:42.956806 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:48:42.957398 13945 external_mini_cluster.cc:1428] Setting key 8bb695dd1d32625baf86c7142dc83d2b
I20260808 11:48:42.960494 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43657
--builtin_ntp_servers=127.13.158.84:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:48:42.967316 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:42.957389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:58305 (local address 127.13.158.126:43657)
0808 11:48:42.957568 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:42.957573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:42.959186 (+ 1613us) server_negotiation.cc:407] Connection header received
0808 11:48:42.961072 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:42.961077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:42.961155 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:42.961269 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:42.962295 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:42.963071 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:42.964602 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:42.965016 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:42.965153 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:42.966517 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:42.966522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:42.966525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:42.966541 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:42.966585 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:42.966595 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:42.966691 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:42.966936 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 11:48:42.967025 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:48:42.968248 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:42.958009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43657 (local address 127.13.158.66:58305)
0808 11:48:42.959037 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:42.959080 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:42.960834 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:42.961501 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:42.961509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:42.961594 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:42.962107 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:42.962122 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:42.963241 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:42.963245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:42.964451 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:42.964464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:42.965275 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:42.965286 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:42.965491 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:42.966200 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:42.966211 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:42.966247 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:42.966321 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:42.966691 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:42.966725 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:42.966735 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 11:48:42.966911 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 11:48:42.967270 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":313,"threads_started":1}
I20260808 11:48:42.969672 7615 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43657
I20260808 11:48:42.970002 7615 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:42.970752 7615 heartbeater.cc:511] Master 127.13.158.126:43657 requested a full tablet report, sending...
I20260808 11:48:42.972255 7157 ts_manager.cc:194] Registered new tserver with Master: 3973fc7eb7cc43e8bbbfc8fad1493224 (127.13.158.66:44117)
I20260808 11:48:42.973106 7157 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.66:58305
W20260808 11:48:43.302011 7619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:43.302280 7619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:43.302429 7619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:43.309351 7619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:43.309442 7619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:43.309603 7619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:48:43.318625 7619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:43657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:43.319950 7619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:43.321810 7619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:43.335246 7629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:43.335290 7632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:43.335518 7628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:43.336220 7619 server_base.cc:1034] running on GCE node
I20260808 11:48:43.336882 7619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:43.337805 7619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:43.339049 7619 hybrid_clock.cc:648] HybridClock initialized: now 1786189723339008 us; error 51 us; skew 500 ppm
I20260808 11:48:43.339402 7619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:43.341610 7619 webserver.cc:460] Webserver started at http://127.13.158.67:44853/ using document root <none> and password file <none>
I20260808 11:48:43.342310 7619 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:43.342450 7619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:43.342856 7619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:43.345898 7619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "56bed01d2c844d6b853a8c7651b5d893"
format_stamp: "Formatted at 2026-08-08 11:48:43 on dist-test-slave-39l8"
server_key: "bf360285190b89bdf41c2ced2f49f785"
server_key_iv: "4d6b291402731a65b1f478cf73c9ed85"
server_key_version: "encryptionkey@0"
I20260808 11:48:43.346678 7619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "56bed01d2c844d6b853a8c7651b5d893"
format_stamp: "Formatted at 2026-08-08 11:48:43 on dist-test-slave-39l8"
server_key: "bf360285190b89bdf41c2ced2f49f785"
server_key_iv: "4d6b291402731a65b1f478cf73c9ed85"
server_key_version: "encryptionkey@0"
I20260808 11:48:43.351971 7619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 11:48:43.355379 7647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:43.356618 7619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:43.356803 7619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "56bed01d2c844d6b853a8c7651b5d893"
format_stamp: "Formatted at 2026-08-08 11:48:43 on dist-test-slave-39l8"
server_key: "bf360285190b89bdf41c2ced2f49f785"
server_key_iv: "4d6b291402731a65b1f478cf73c9ed85"
server_key_version: "encryptionkey@0"
I20260808 11:48:43.357015 7619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:43.381906 7619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:43.382697 7619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:43.383070 7619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:43.383999 7619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:43.385417 7619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:43.385550 7619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:43.385699 7619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:43.385797 7619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:43.420397 7619 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:44111
I20260808 11:48:43.420507 7807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:44111 every 8 connection(s)
I20260808 11:48:43.421773 7619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:48:43.422358 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 7619
I20260808 11:48:43.422591 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:48:43.423228 13945 external_mini_cluster.cc:1428] Setting key 951c28af3321a397de3606c70563ddaf
I20260808 11:48:43.433862 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.424378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:38615 (local address 127.13.158.126:43657)
0808 11:48:43.424597 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.424602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.426264 (+ 1662us) server_negotiation.cc:407] Connection header received
0808 11:48:43.428134 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.428138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.428204 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.428286 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:43.429238 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.429955 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.431467 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.431837 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.431926 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.433193 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:43.433197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:43.433201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:43.433218 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:43.433264 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:43.433275 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:43.433342 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.433551 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.433625 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 11:48:43.434863 7814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.425137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:43657 (local address 127.13.158.67:38615)
0808 11:48:43.426093 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:43.426130 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:43.427883 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:43.428480 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:43.428488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:43.428564 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:43.429091 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.429110 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.430081 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.430085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.431343 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.431355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.432029 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.432034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.432270 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.432906 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:43.432908 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:43.432950 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:43.433020 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:43.433362 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:43.433369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:43.433375 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:48:43.433508 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 11:48:43.433757 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":335,"threads_started":1}
I20260808 11:48:43.436269 7809 heartbeater.cc:347] Connected to a master server at 127.13.158.126:43657
I20260808 11:48:43.436584 7809 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:43.437372 7809 heartbeater.cc:511] Master 127.13.158.126:43657 requested a full tablet report, sending...
I20260808 11:48:43.438917 7157 ts_manager.cc:194] Registered new tserver with Master: 56bed01d2c844d6b853a8c7651b5d893 (127.13.158.67:44111)
I20260808 11:48:43.439656 7157 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.67:38615
I20260808 11:48:43.450867 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:48:43.461001 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.455081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50110 (local address 127.13.158.126:43657)
0808 11:48:43.455245 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.455249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.455728 (+ 479us) server_negotiation.cc:407] Connection header received
0808 11:48:43.455852 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.455856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.455924 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.456001 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:43.456645 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.457303 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.458816 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.459215 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.459321 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.460309 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:43.460313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:43.460316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:43.460332 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:43.460361 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:43.460372 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:43.460454 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.460617 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.460685 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:48:43.464538 7157 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:48:43.471475 7157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:48:43.495126 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.484200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:44117 (local address 127.0.0.1:40794)
0808 11:48:43.486367 (+ 2167us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:43.486381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:43.486521 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:43.487258 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:43.487270 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:43.487314 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:43.487988 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.488008 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.490173 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.490179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.492318 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.492331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.494077 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.494083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.494274 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.494332 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 11:48:43.494490 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 11:48:43.494686 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2018,"mutex_wait_us":196,"thread_start_us":3538,"threads_started":1}
I20260808 11:48:43.495103 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.484570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.13.158.66:44117)
0808 11:48:43.485484 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.485496 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.486509 (+ 1013us) server_negotiation.cc:407] Connection header received
0808 11:48:43.486673 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.486681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.486977 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.487189 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:43.488400 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.489983 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.492520 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.493870 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.494018 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.494270 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.494564 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.494754 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":610,"threads_started":1}
I20260808 11:48:43.495985 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.483687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:44111 (local address 127.0.0.1:40086)
0808 11:48:43.485884 (+ 2197us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:43.485923 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:43.486143 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:43.487507 (+ 1364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:43.487512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:43.487545 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:43.488147 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.488163 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.490283 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.490295 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.492938 (+ 2643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.492953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.495139 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.495147 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.495359 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.495388 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 11:48:43.495468 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 11:48:43.495594 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1925,"mutex_wait_us":144,"thread_start_us":2538,"threads_started":1}
I20260808 11:48:43.496830 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.483959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40086 (local address 127.13.158.67:44111)
0808 11:48:43.486779 (+ 2820us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.486786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.486818 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:48:43.486916 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.486924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.487143 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.487315 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:43.488379 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.490134 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.493172 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.494557 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.494696 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.494990 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.496268 (+ 1278us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.496441 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":926,"thread_start_us":728,"threads_started":1}
I20260808 11:48:43.496833 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.484387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54552)
0808 11:48:43.488071 (+ 3684us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:43.488092 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:43.488220 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:43.488876 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:43.488881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:43.488908 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:43.489233 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.489244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.491264 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.491269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.493513 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.493527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.496045 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.496052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.496249 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.496280 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:48:43.496361 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 11:48:43.496469 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3530,"mutex_wait_us":83,"thread_start_us":1100,"threads_started":1}
I20260808 11:48:43.497575 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.484216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.13.158.65:33019)
0808 11:48:43.485912 (+ 1696us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.485922 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.488212 (+ 2290us) server_negotiation.cc:407] Connection header received
0808 11:48:43.488391 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.488399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.488639 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.488851 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:43.489495 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.491099 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.493726 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.495157 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.495651 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.495911 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.497051 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.497232 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1544,"thread_start_us":1414,"threads_started":1}
I20260808 11:48:43.498875 7526 tablet_service.cc:1467] Processing CreateTablet for tablet 102b0e820a9b4d8fb92fd3c08f0b3add (DEFAULT_TABLE table=test-table [id=513dc6d5e3074098a6a9351ddd770c7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:43.500059 7526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102b0e820a9b4d8fb92fd3c08f0b3add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:43.500379 7718 tablet_service.cc:1467] Processing CreateTablet for tablet 102b0e820a9b4d8fb92fd3c08f0b3add (DEFAULT_TABLE table=test-table [id=513dc6d5e3074098a6a9351ddd770c7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:43.500808 7334 tablet_service.cc:1467] Processing CreateTablet for tablet 102b0e820a9b4d8fb92fd3c08f0b3add (DEFAULT_TABLE table=test-table [id=513dc6d5e3074098a6a9351ddd770c7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:43.501816 7718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102b0e820a9b4d8fb92fd3c08f0b3add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:43.502113 7334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102b0e820a9b4d8fb92fd3c08f0b3add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:43.503032 7427 heartbeater.cc:503] Master 127.13.158.126:43657 was elected leader, sending a full tablet report...
I20260808 11:48:43.512759 7840 tablet_bootstrap.cc:492] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Bootstrap starting.
I20260808 11:48:43.516371 7840 tablet_bootstrap.cc:654] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:43.516973 7841 tablet_bootstrap.cc:492] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Bootstrap starting.
I20260808 11:48:43.517659 7840 log.cc:824] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:43.519430 7845 tablet_bootstrap.cc:492] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: Bootstrap starting.
I20260808 11:48:43.520771 7841 tablet_bootstrap.cc:654] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:43.521009 7840 tablet_bootstrap.cc:492] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: No bootstrap required, opened a new log
I20260808 11:48:43.521278 7840 ts_tablet_manager.cc:1397] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 11:48:43.521951 7841 log.cc:824] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:43.522884 7845 tablet_bootstrap.cc:654] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:43.524044 7845 log.cc:824] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:43.525372 7841 tablet_bootstrap.cc:492] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: No bootstrap required, opened a new log
I20260808 11:48:43.525691 7841 ts_tablet_manager.cc:1397] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260808 11:48:43.529667 7840 raft_consensus.cc:348] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.530140 7840 raft_consensus.cc:374] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:43.530273 7840 raft_consensus.cc:729] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3973fc7eb7cc43e8bbbfc8fad1493224, State: Initialized, Role: FOLLOWER
I20260808 11:48:43.530915 7840 consensus_queue.cc:260] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.531589 7845 tablet_bootstrap.cc:492] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: No bootstrap required, opened a new log
I20260808 11:48:43.531843 7845 ts_tablet_manager.cc:1397] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260808 11:48:43.532060 7615 heartbeater.cc:503] Master 127.13.158.126:43657 was elected leader, sending a full tablet report...
I20260808 11:48:43.532697 7840 ts_tablet_manager.cc:1428] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260808 11:48:43.537149 7841 raft_consensus.cc:348] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.537549 7841 raft_consensus.cc:374] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:43.537655 7841 raft_consensus.cc:729] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bed01d2c844d6b853a8c7651b5d893, State: Initialized, Role: FOLLOWER
I20260808 11:48:43.538233 7841 consensus_queue.cc:260] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.539279 7809 heartbeater.cc:503] Master 127.13.158.126:43657 was elected leader, sending a full tablet report...
I20260808 11:48:43.539917 7841 ts_tablet_manager.cc:1428] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260808 11:48:43.541603 7845 raft_consensus.cc:348] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.541961 7845 raft_consensus.cc:374] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:43.542068 7845 raft_consensus.cc:729] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2e9ff52c664f16979d650938d7772c, State: Initialized, Role: FOLLOWER
I20260808 11:48:43.542631 7845 consensus_queue.cc:260] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.544481 7845 ts_tablet_manager.cc:1428] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
W20260808 11:48:43.673672 7810 tablet.cc:2369] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:48:43.707191 7617 tablet.cc:2369] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:48:43.728827 7428 tablet.cc:2369] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:43.729066 7852 raft_consensus.cc:482] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:43.729449 7852 raft_consensus.cc:504] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.731241 7852 leader_election.cc:288] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2e9ff52c664f16979d650938d7772c (127.13.158.65:33019), 3973fc7eb7cc43e8bbbfc8fad1493224 (127.13.158.66:44117)
I20260808 11:48:43.737238 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.731763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:60069 (local address 127.13.158.65:33019)
0808 11:48:43.731943 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.731949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.731969 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:48:43.732256 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.732262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.732366 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.732476 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:43.732896 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.733588 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.735409 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.736346 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.736515 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.736587 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.736860 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.736973 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:48:43.737274 7814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.731538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.13.158.67:60069)
0808 11:48:43.731840 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:43.731865 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:43.732032 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:43.732533 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:43.732536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:43.732560 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:43.732718 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.732726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.733741 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.733747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.735253 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.735265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.736495 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.736501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.736690 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.736713 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:48:43.736819 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 11:48:43.736952 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 11:48:43.737823 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.731802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:41753 (local address 127.13.158.66:44117)
0808 11:48:43.731976 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.731981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.732648 (+ 667us) server_negotiation.cc:407] Connection header received
0808 11:48:43.732872 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.732877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.732948 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.733051 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:43.733587 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.734231 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.736063 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.736868 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.737049 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.737112 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.737465 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.737581 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:48:43.738458 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.731605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:44117 (local address 127.13.158.67:41753)
0808 11:48:43.732551 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:43.732570 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:43.732699 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:43.733081 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:43.733085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:43.733108 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:43.733429 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.733438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.734386 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.734391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.735885 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:43.735897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.737065 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.737072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:43.737244 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.737270 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 11:48:43.738013 (+ 743us) client_negotiation.cc:240] Negotiation successful
0808 11:48:43.738149 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":612,"threads_started":1}
I20260808 11:48:43.738371 7358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b0e820a9b4d8fb92fd3c08f0b3add" candidate_uuid: "56bed01d2c844d6b853a8c7651b5d893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2e9ff52c664f16979d650938d7772c" is_pre_election: true
I20260808 11:48:43.738951 7358 raft_consensus.cc:2454] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56bed01d2c844d6b853a8c7651b5d893 in term 0.
I20260808 11:48:43.739598 7649 leader_election.cc:302] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2e9ff52c664f16979d650938d7772c, 56bed01d2c844d6b853a8c7651b5d893; no voters:
I20260808 11:48:43.739527 7551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b0e820a9b4d8fb92fd3c08f0b3add" candidate_uuid: "56bed01d2c844d6b853a8c7651b5d893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" is_pre_election: true
I20260808 11:48:43.739977 7852 raft_consensus.cc:2781] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:48:43.739986 7551 raft_consensus.cc:2454] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56bed01d2c844d6b853a8c7651b5d893 in term 0.
I20260808 11:48:43.740136 7852 raft_consensus.cc:482] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:48:43.740294 7852 raft_consensus.cc:3037] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:43.742558 7852 raft_consensus.cc:504] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.743266 7852 leader_election.cc:288] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2e9ff52c664f16979d650938d7772c (127.13.158.65:33019), 3973fc7eb7cc43e8bbbfc8fad1493224 (127.13.158.66:44117)
I20260808 11:48:43.743618 7358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b0e820a9b4d8fb92fd3c08f0b3add" candidate_uuid: "56bed01d2c844d6b853a8c7651b5d893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2e9ff52c664f16979d650938d7772c"
I20260808 11:48:43.743820 7358 raft_consensus.cc:3037] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:43.743750 7551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102b0e820a9b4d8fb92fd3c08f0b3add" candidate_uuid: "56bed01d2c844d6b853a8c7651b5d893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224"
I20260808 11:48:43.744002 7551 raft_consensus.cc:3037] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:43.746122 7358 raft_consensus.cc:2454] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bed01d2c844d6b853a8c7651b5d893 in term 1.
I20260808 11:48:43.746240 7551 raft_consensus.cc:2454] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bed01d2c844d6b853a8c7651b5d893 in term 1.
I20260808 11:48:43.746544 7649 leader_election.cc:302] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2e9ff52c664f16979d650938d7772c, 56bed01d2c844d6b853a8c7651b5d893; no voters:
I20260808 11:48:43.746829 7852 raft_consensus.cc:2781] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:43.747279 7852 raft_consensus.cc:686] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 1 LEADER]: Becoming Leader. State: Replica: 56bed01d2c844d6b853a8c7651b5d893, State: Running, Role: LEADER
I20260808 11:48:43.747817 7852 consensus_queue.cc:237] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } }
I20260808 11:48:43.752550 7157 catalog_manager.cc:5495] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56bed01d2c844d6b853a8c7651b5d893 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "56bed01d2c844d6b853a8c7651b5d893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bed01d2c844d6b853a8c7651b5d893" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 44111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 } health_report { overall_health: UNKNOWN } } }
I20260808 11:48:43.790143 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.784986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40088 (local address 127.13.158.67:44111)
0808 11:48:43.785182 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.785187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.785270 (+ 83us) server_negotiation.cc:407] Connection header received
0808 11:48:43.785438 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.785442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.785519 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.785649 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:48:43.785992 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.786744 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.788061 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.788521 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.788612 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.789105 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:48:43.789687 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:48:43.789796 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.789901 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.789960 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:48:43.798117 7358 raft_consensus.cc:1261] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 1 FOLLOWER]: Refusing update from remote peer 56bed01d2c844d6b853a8c7651b5d893: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:43.798122 7551 raft_consensus.cc:1261] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 1 FOLLOWER]: Refusing update from remote peer 56bed01d2c844d6b853a8c7651b5d893: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:43.799119 7856 consensus_queue.cc:1035] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2e9ff52c664f16979d650938d7772c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 33019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:43.799479 7852 consensus_queue.cc:1035] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3973fc7eb7cc43e8bbbfc8fad1493224" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 44117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:43.812722 7861 mvcc.cc:205] Tried to move back new op lower bound from 7316233108667568128 to 7316233108475428864. Current Snapshot: MvccSnapshot[applied={T|T < 7316233108667568128}]
I20260808 11:48:43.814010 7860 mvcc.cc:205] Tried to move back new op lower bound from 7316233108667568128 to 7316233108475428864. Current Snapshot: MvccSnapshot[applied={T|T < 7316233108667568128}]
I20260808 11:48:43.823740 7156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:43.824044 7156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:43.827586 7156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33fc3e32255a486babe0bfb8964787cf: Sending DeleteTablet for 3 replicas of tablet 102b0e820a9b4d8fb92fd3c08f0b3add
I20260808 11:48:43.828676 7718 tablet_service.cc:1514] Processing DeleteTablet for tablet 102b0e820a9b4d8fb92fd3c08f0b3add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:43 UTC) from {username='slave'} at 127.0.0.1:40086
I20260808 11:48:43.828990 7526 tablet_service.cc:1514] Processing DeleteTablet for tablet 102b0e820a9b4d8fb92fd3c08f0b3add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:43 UTC) from {username='slave'} at 127.0.0.1:40794
I20260808 11:48:43.829326 7334 tablet_service.cc:1514] Processing DeleteTablet for tablet 102b0e820a9b4d8fb92fd3c08f0b3add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:43 UTC) from {username='slave'} at 127.0.0.1:54552
I20260808 11:48:43.830314 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 7243
I20260808 11:48:43.835222 7877 tablet_replica.cc:330] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: stopping tablet replica
I20260808 11:48:43.835759 7877 raft_consensus.cc:2229] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:43.836308 7877 raft_consensus.cc:2258] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:43.844045 7871 tablet_replica.cc:330] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: stopping tablet replica
I20260808 11:48:43.844655 7871 raft_consensus.cc:2229] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:48:43.844930 7877 ts_tablet_manager.cc:1905] T 102b0e820a9b4d8fb92fd3c08f0b3add P 2b2e9ff52c664f16979d650938d7772c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:43.845429 7871 raft_consensus.cc:2258] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:43.847466 7871 ts_tablet_manager.cc:1905] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:43.848594 7872 tablet_replica.cc:330] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: stopping tablet replica
I20260808 11:48:43.848994 7872 raft_consensus.cc:2229] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:43.849469 7872 raft_consensus.cc:2258] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:43.851068 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:43.832538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54566 (local address 127.13.158.65:33019)
0808 11:48:43.832771 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:43.832781 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:43.841211 (+ 8430us) server_negotiation.cc:407] Connection header received
0808 11:48:43.841406 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:43.841415 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:43.841558 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:43.841695 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:43.842472 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.844010 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.846651 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:43.847675 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:43.847803 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:43.849455 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:43.849470 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:43.849482 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:43.850272 (+ 790us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:43.850313 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:43.850332 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:43.850457 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:43.850553 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 11:48:43.850652 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:48:43.851831 7422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:43.854985 7871 ts_tablet_manager.cc:1918] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:43.855212 7871 log.cc:1196] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/102b0e820a9b4d8fb92fd3c08f0b3add
I20260808 11:48:43.855578 7872 ts_tablet_manager.cc:1905] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:43.855970 7871 ts_tablet_manager.cc:1939] T 102b0e820a9b4d8fb92fd3c08f0b3add P 56bed01d2c844d6b853a8c7651b5d893: Deleting consensus metadata
I20260808 11:48:43.857856 7139 catalog_manager.cc:4841] TS 56bed01d2c844d6b853a8c7651b5d893 (127.13.158.67:44111): tablet 102b0e820a9b4d8fb92fd3c08f0b3add (table test-table [id=513dc6d5e3074098a6a9351ddd770c7c]) successfully deleted
I20260808 11:48:43.863524 7872 ts_tablet_manager.cc:1918] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:43.863692 7872 log.cc:1196] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/102b0e820a9b4d8fb92fd3c08f0b3add
I20260808 11:48:43.864187 7872 ts_tablet_manager.cc:1939] T 102b0e820a9b4d8fb92fd3c08f0b3add P 3973fc7eb7cc43e8bbbfc8fad1493224: Deleting consensus metadata
I20260808 11:48:43.865718 7137 catalog_manager.cc:4841] TS 3973fc7eb7cc43e8bbbfc8fad1493224 (127.13.158.66:44117): tablet 102b0e820a9b4d8fb92fd3c08f0b3add (table test-table [id=513dc6d5e3074098a6a9351ddd770c7c]) successfully deleted
I20260808 11:48:43.978564 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 7243
W20260808 11:48:44.007688 7138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.158.65:33019 (error 108)
I20260808 11:48:44.007845 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 7432
I20260808 11:48:44.010035 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.009255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54578)
0808 11:48:44.009693 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:44.009830 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":235,"threads_started":1}
W20260808 11:48:44.010335 7138 catalog_manager.cc:4584] TS 2b2e9ff52c664f16979d650938d7772c (127.13.158.65:33019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 102b0e820a9b4d8fb92fd3c08f0b3add: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
I20260808 11:48:44.018078 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.009050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.13.158.66:44117)
0808 11:48:44.009281 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:44.009288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:44.009307 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:48:44.009373 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:44.009379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:44.009487 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:44.009589 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:44.010231 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:44.011212 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:44.013156 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:44.013947 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:44.014094 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:44.015697 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:44.015711 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:44.015732 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:44.016608 (+ 876us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:44.016729 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:44.016762 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:44.016878 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:44.017518 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 11:48:44.017727 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 11:48:44.019104 7608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:44.064170 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.063615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54590)
0808 11:48:44.063871 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:44.063986 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 11:48:44.065411 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.064966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54594)
0808 11:48:44.065162 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:44.065248 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 11:48:44.110462 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 7432
I20260808 11:48:44.126283 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.125595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54610)
0808 11:48:44.125948 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:44.126071 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 11:48:44.127573 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.127119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54616)
0808 11:48:44.127307 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:44.127412 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 11:48:44.136514 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 7619
I20260808 11:48:44.144790 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.137354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.13.158.67:44111)
0808 11:48:44.137541 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:44.137547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:44.137569 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:48:44.137748 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:44.137753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:44.137853 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:44.137975 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:44.138366 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:44.139551 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:44.140803 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:44.141490 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:44.141634 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:44.143077 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:44.143087 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:44.143104 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:44.143972 (+ 868us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:44.144038 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:44.144063 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:44.144178 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:44.144288 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 11:48:44.144384 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:48:44.145593 7802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:44.208511 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.207942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54618)
0808 11:48:44.208200 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:44.208324 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 11:48:44.209723 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.209222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54632)
0808 11:48:44.209460 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:44.209545 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260808 11:48:44.268854 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 7619
I20260808 11:48:44.297317 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 7116
I20260808 11:48:44.298441 7197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:44.407243 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:44.406468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:33019 (local address 127.0.0.1:54646)
0808 11:48:44.406866 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:44.407046 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:33019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260808 11:48:44.407265 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 7116
2026-08-08T11:48:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3064 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:48:44 dist-test-slave-39l8 krb5kdc[7890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:48:44 dist-test-slave-39l8 krb5kdc[7890](info): set up 2 sockets
Aug 08 11:48:44 dist-test-slave-39l8 krb5kdc[7890](info): commencing operation
krb5kdc: starting...
W20260808 11:48:45.710004 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:48:45 dist-test-slave-39l8 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:48:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:48:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:48:46.022897 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:45271
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:45271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:46.358605 7905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:46.358906 7905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:46.359076 7905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:46.366091 7905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:48:46.366185 7905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:46.366231 7905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:46.366348 7905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:48:46.366410 7905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:48:46.376012 7905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:45271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:45271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:46.377339 7905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:46.378623 7905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:46.391119 7913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:46.391016 7914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:46.391040 7919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:46.391850 7905 server_base.cc:1034] running on GCE node
I20260808 11:48:46.392709 7905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:46.394116 7905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:46.395347 7905 hybrid_clock.cc:648] HybridClock initialized: now 1786189726395302 us; error 36 us; skew 500 ppm
I20260808 11:48:46.395759 7905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:46 dist-test-slave-39l8 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189726, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:46.399206 7905 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:48:46.400825 7905 webserver.cc:460] Webserver started at http://127.13.158.126:42451/ using document root <none> and password file <none>
I20260808 11:48:46.401729 7905 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:46.401907 7905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:46.402307 7905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:46.405361 7905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "dddf567a91f44c8db1ce67049d68a09e"
format_stamp: "Formatted at 2026-08-08 11:48:46 on dist-test-slave-39l8"
server_key: "beefca6035863493ce362138d49324b4"
server_key_iv: "1eeca23e8ef45c8415d4d33c92584d15"
server_key_version: "encryptionkey@0"
I20260808 11:48:46.406157 7905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "dddf567a91f44c8db1ce67049d68a09e"
format_stamp: "Formatted at 2026-08-08 11:48:46 on dist-test-slave-39l8"
server_key: "beefca6035863493ce362138d49324b4"
server_key_iv: "1eeca23e8ef45c8415d4d33c92584d15"
server_key_version: "encryptionkey@0"
I20260808 11:48:46.411514 7905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260808 11:48:46.415199 7927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:46.416489 7905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:48:46.416688 7905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "dddf567a91f44c8db1ce67049d68a09e"
format_stamp: "Formatted at 2026-08-08 11:48:46 on dist-test-slave-39l8"
server_key: "beefca6035863493ce362138d49324b4"
server_key_iv: "1eeca23e8ef45c8415d4d33c92584d15"
server_key_version: "encryptionkey@0"
I20260808 11:48:46.416904 7905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:46.438215 7905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:46.442667 7905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:46.443006 7905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:46.461928 7905 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:45271
I20260808 11:48:46.461977 7997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:45271 every 8 connection(s)
I20260808 11:48:46.463371 7905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:48:46.467381 7998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:46.473678 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 7905
I20260808 11:48:46.473905 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:48:46.474380 13945 external_mini_cluster.cc:1428] Setting key 94c5e04a1fac1eb9e41c0b12feb90e9e
I20260808 11:48:46.480320 7998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e: Bootstrap starting.
I20260808 11:48:46.484073 7998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:46.485271 7998 log.cc:824] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:46.488483 7998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e: No bootstrap required, opened a new log
Aug 08 11:48:46 dist-test-slave-39l8 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:46.498471 7998 raft_consensus.cc:348] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf567a91f44c8db1ce67049d68a09e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45271 } }
I20260808 11:48:46.499011 7998 raft_consensus.cc:374] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:46.499150 7998 raft_consensus.cc:729] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dddf567a91f44c8db1ce67049d68a09e, State: Initialized, Role: FOLLOWER
I20260808 11:48:46.499845 7998 consensus_queue.cc:260] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf567a91f44c8db1ce67049d68a09e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45271 } }
I20260808 11:48:46.500124 7998 raft_consensus.cc:388] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:48:46.500269 7998 raft_consensus.cc:482] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:48:46.500452 7998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:46.501421 8008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:46.476767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.13.158.126:45271)
0808 11:48:46.477677 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:46.477698 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:46.477746 (+ 48us) server_negotiation.cc:407] Connection header received
0808 11:48:46.480108 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:46.480135 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:46.480623 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:46.481072 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:46.482619 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:46.484619 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:46.486440 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:46.487421 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:46.487593 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:46.492335 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:46.492346 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:46.492366 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:46.492412 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:46.496566 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:46.497325 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:46.497334 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:46.497343 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:46.497563 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:46.497959 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:46.497964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:46.497967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:46.498687 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:46.498987 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:46.499576 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 11:48:46.500007 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":420,"threads_started":1}
I20260808 11:48:46.503547 7998 raft_consensus.cc:504] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf567a91f44c8db1ce67049d68a09e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45271 } }
I20260808 11:48:46.504107 7998 leader_election.cc:302] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dddf567a91f44c8db1ce67049d68a09e; no voters:
I20260808 11:48:46.504729 7998 leader_election.cc:288] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:48:46.504859 8011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:46.505592 8011 raft_consensus.cc:686] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [term 1 LEADER]: Becoming Leader. State: Replica: dddf567a91f44c8db1ce67049d68a09e, State: Running, Role: LEADER
I20260808 11:48:46.506201 8011 consensus_queue.cc:237] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf567a91f44c8db1ce67049d68a09e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45271 } }
I20260808 11:48:46.506898 7998 sys_catalog.cc:564] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:48:46.509171 8014 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dddf567a91f44c8db1ce67049d68a09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf567a91f44c8db1ce67049d68a09e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45271 } } }
I20260808 11:48:46.509279 8015 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dddf567a91f44c8db1ce67049d68a09e. Latest consensus state: current_term: 1 leader_uuid: "dddf567a91f44c8db1ce67049d68a09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf567a91f44c8db1ce67049d68a09e" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 45271 } } }
I20260808 11:48:46.509560 8014 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:46.509647 8015 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:46.510543 8020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:48:46.516956 8020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:48:46.526261 8020 catalog_manager.cc:1347] Generated new cluster ID: ebebbeb771cf420088ef79e5adeaaa1b
I20260808 11:48:46.526376 8020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:48:46.535573 8020 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:48:46.537288 8020 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:48:46.547901 8020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e: Generated new TSK 0
I20260808 11:48:46.548692 8020 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:48:46.639954 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45271
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:47.009866 8042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:47.010131 8042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:47.010303 8042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:47.017729 8042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:47.017824 8042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:47.018044 8042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:48:47.027495 8042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:47.028684 8042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:47.029935 8042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:47.042798 8048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:47.042774 8049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:47.043058 8051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:47.043524 8042 server_base.cc:1034] running on GCE node
I20260808 11:48:47.044059 8042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:47.044978 8042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:47.046208 8042 hybrid_clock.cc:648] HybridClock initialized: now 1786189727046124 us; error 73 us; skew 500 ppm
I20260808 11:48:47.046497 8042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:47 dist-test-slave-39l8 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189727, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:47.054899 8042 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:48:47.056391 8042 webserver.cc:460] Webserver started at http://127.13.158.65:36865/ using document root <none> and password file <none>
I20260808 11:48:47.056986 8042 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:47.057085 8042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:47.057332 8042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:47.060105 8042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "242f137c49cd4220853c7bbe60e9b933"
format_stamp: "Formatted at 2026-08-08 11:48:47 on dist-test-slave-39l8"
server_key: "098c777654d7029ac25a66925593748b"
server_key_iv: "d4f3e741419e435da43faff1df6df6a6"
server_key_version: "encryptionkey@0"
I20260808 11:48:47.060806 8042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "242f137c49cd4220853c7bbe60e9b933"
format_stamp: "Formatted at 2026-08-08 11:48:47 on dist-test-slave-39l8"
server_key: "098c777654d7029ac25a66925593748b"
server_key_iv: "d4f3e741419e435da43faff1df6df6a6"
server_key_version: "encryptionkey@0"
I20260808 11:48:47.065757 8042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:48:47.069029 8061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:47.070240 8042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 11:48:47.070366 8042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "242f137c49cd4220853c7bbe60e9b933"
format_stamp: "Formatted at 2026-08-08 11:48:47 on dist-test-slave-39l8"
server_key: "098c777654d7029ac25a66925593748b"
server_key_iv: "d4f3e741419e435da43faff1df6df6a6"
server_key_version: "encryptionkey@0"
I20260808 11:48:47.070519 8042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:47.107538 8042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:47.111471 8042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:47.111717 8042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:47.112773 8042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:47.114115 8042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:47.114187 8042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:47.114271 8042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:47.114315 8042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:47.148403 8042 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:41411
I20260808 11:48:47.148453 8231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:41411 every 8 connection(s)
I20260808 11:48:47.149725 8042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:48:47.154753 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 8042
I20260808 11:48:47.154951 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:48:47.155524 13945 external_mini_cluster.cc:1428] Setting key 23a65d5c7efd28b0e8704cb87fb95ea1
Aug 08 11:48:47 dist-test-slave-39l8 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189727, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:47.171890 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:47.152291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:33435 (local address 127.13.158.126:45271)
0808 11:48:47.152768 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:47.152773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:47.153522 (+ 749us) server_negotiation.cc:407] Connection header received
0808 11:48:47.155552 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:47.155559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:47.155656 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:47.155794 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:47.157740 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:47.158800 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:47.160291 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:47.160972 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:47.161116 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:47.164698 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:47.164714 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:47.164721 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:47.164773 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:47.168447 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:47.169232 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:47.169239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:47.169245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:47.169433 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:47.169841 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:47.169848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:47.169854 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:47.170217 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:47.170367 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:47.171145 (+ 778us) server_negotiation.cc:299] Negotiation successful
0808 11:48:47.171407 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":258,"threads_started":1}
I20260808 11:48:47.172670 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:47.152669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45271 (local address 127.13.158.65:33435)
0808 11:48:47.153352 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:47.153392 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:47.155247 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:47.155992 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:47.156000 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:47.156693 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:47.157524 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:47.157554 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:47.158949 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:47.158953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:47.160129 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:47.160151 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:47.161210 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:47.161219 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:47.161522 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:47.162217 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:47.162238 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:47.164475 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:47.168671 (+ 4196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:47.168678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:47.168690 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:47.169098 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:47.169545 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:47.169548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:47.169550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:47.169727 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:47.170340 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:47.170346 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:47.170638 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 11:48:47.171004 (+ 366us) client_negotiation.cc:240] Negotiation successful
0808 11:48:47.171381 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":293,"threads_started":1}
I20260808 11:48:47.174405 8232 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45271
I20260808 11:48:47.174748 8232 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:47.175657 8232 heartbeater.cc:511] Master 127.13.158.126:45271 requested a full tablet report, sending...
I20260808 11:48:47.178117 7955 ts_manager.cc:194] Registered new tserver with Master: 242f137c49cd4220853c7bbe60e9b933 (127.13.158.65:41411)
I20260808 11:48:47.180248 7955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:33435
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:48:47.226486 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45271
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:47.560259 8240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:47.560596 8240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:47.560767 8240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:47.568064 8240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:47.568226 8240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:47.568418 8240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:48:47.578202 8240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:47.579455 8240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:47.580786 8240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:47.594288 8249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:47.594354 8247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:47.594378 8252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:47.594933 8240 server_base.cc:1034] running on GCE node
I20260808 11:48:47.595578 8240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:47.596472 8240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:47.597712 8240 hybrid_clock.cc:648] HybridClock initialized: now 1786189727597660 us; error 39 us; skew 500 ppm
I20260808 11:48:47.598032 8240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:47 dist-test-slave-39l8 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189727, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:47.601030 8240 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:48:47.602612 8240 webserver.cc:460] Webserver started at http://127.13.158.66:37515/ using document root <none> and password file <none>
I20260808 11:48:47.603338 8240 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:47.603444 8240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:47.603706 8240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:47.606498 8240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "13c263c0b65b44f6b3ad216f50e97c15"
format_stamp: "Formatted at 2026-08-08 11:48:47 on dist-test-slave-39l8"
server_key: "132c8cd5aa7a281bd5b03e53a21a1ff9"
server_key_iv: "e5c3247a9fc34a9043d539850bf34c90"
server_key_version: "encryptionkey@0"
I20260808 11:48:47.607261 8240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "13c263c0b65b44f6b3ad216f50e97c15"
format_stamp: "Formatted at 2026-08-08 11:48:47 on dist-test-slave-39l8"
server_key: "132c8cd5aa7a281bd5b03e53a21a1ff9"
server_key_iv: "e5c3247a9fc34a9043d539850bf34c90"
server_key_version: "encryptionkey@0"
I20260808 11:48:47.612512 8240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:48:47.615967 8260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:47.617242 8240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:48:47.617429 8240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "13c263c0b65b44f6b3ad216f50e97c15"
format_stamp: "Formatted at 2026-08-08 11:48:47 on dist-test-slave-39l8"
server_key: "132c8cd5aa7a281bd5b03e53a21a1ff9"
server_key_iv: "e5c3247a9fc34a9043d539850bf34c90"
server_key_version: "encryptionkey@0"
I20260808 11:48:47.617599 8240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:47.640154 8240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:47.644115 8240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:47.644416 8240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:47.645421 8240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:47.646772 8240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:47.646842 8240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:47.647001 8240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:47.647060 8240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:47.681823 8240 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:35133
I20260808 11:48:47.681921 8425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:35133 every 8 connection(s)
I20260808 11:48:47.683262 8240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:48:47.686604 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 8240
I20260808 11:48:47.686834 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:48:47.687472 13945 external_mini_cluster.cc:1428] Setting key 3906a6ff80500231ff9a1479883035d3
Aug 08 11:48:47 dist-test-slave-39l8 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189727, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:47.705955 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:47.685931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:56887 (local address 127.13.158.126:45271)
0808 11:48:47.686411 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:47.686417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:47.687506 (+ 1089us) server_negotiation.cc:407] Connection header received
0808 11:48:47.689585 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:47.689590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:47.689682 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:47.689795 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:47.691338 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:47.692559 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:47.694511 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:47.695197 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:47.695354 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:47.699767 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:47.699772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:47.699776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:47.699808 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:47.702333 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:47.703288 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:47.703292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:47.703294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:47.703417 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:47.703912 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:47.703919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:47.703924 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:47.704328 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:47.704486 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:47.705193 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 11:48:47.705471 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":262,"threads_started":1}
I20260808 11:48:47.706557 8430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:47.686449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45271 (local address 127.13.158.66:56887)
0808 11:48:47.687335 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:47.687392 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:47.689247 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:47.689982 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:47.689990 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:47.690534 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:47.691156 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:47.691173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:47.692704 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:47.692709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:47.694323 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:47.694338 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:47.695485 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:47.695496 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:47.695709 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:47.696565 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:47.696620 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:47.699536 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:47.702478 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:47.702489 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:47.702506 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:47.703168 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:47.703598 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:47.703601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:47.703603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:47.703781 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:47.704462 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:47.704468 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:47.704786 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 11:48:47.705028 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 11:48:47.705390 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":438,"threads_started":1}
I20260808 11:48:47.708129 8427 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45271
I20260808 11:48:47.708452 8427 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:47.709347 8427 heartbeater.cc:511] Master 127.13.158.126:45271 requested a full tablet report, sending...
I20260808 11:48:47.710944 7955 ts_manager.cc:194] Registered new tserver with Master: 13c263c0b65b44f6b3ad216f50e97c15 (127.13.158.66:35133)
I20260808 11:48:47.711845 7955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:56887
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:48:47.758558 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45271
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:39349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:48.099488 8435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:48.099754 8435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:48.099920 8435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:48.107316 8435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:48.107422 8435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:48.107640 8435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:48:48.117447 8435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:39349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:45271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:48.118775 8435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:48.120203 8435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:48.133524 8441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:48.133570 8445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:48.133595 8442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:48.134342 8435 server_base.cc:1034] running on GCE node
I20260808 11:48:48.135020 8435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:48.135931 8435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:48.137179 8435 hybrid_clock.cc:648] HybridClock initialized: now 1786189728137143 us; error 64 us; skew 500 ppm
I20260808 11:48:48.137502 8435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:48 dist-test-slave-39l8 krb5kdc[7890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189728, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:48.140477 8435 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:48:48.141948 8435 webserver.cc:460] Webserver started at http://127.13.158.67:43129/ using document root <none> and password file <none>
I20260808 11:48:48.142665 8435 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:48.142876 8435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:48.143220 8435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:48.146025 8435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "6c4da626004c4351b5a043a74ec3fb28"
format_stamp: "Formatted at 2026-08-08 11:48:48 on dist-test-slave-39l8"
server_key: "b55afc781f93ca9a439ff53d91dcb610"
server_key_iv: "44b303454be161b756de51d8e0b89904"
server_key_version: "encryptionkey@0"
I20260808 11:48:48.146946 8435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "6c4da626004c4351b5a043a74ec3fb28"
format_stamp: "Formatted at 2026-08-08 11:48:48 on dist-test-slave-39l8"
server_key: "b55afc781f93ca9a439ff53d91dcb610"
server_key_iv: "44b303454be161b756de51d8e0b89904"
server_key_version: "encryptionkey@0"
I20260808 11:48:48.152209 8435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:48:48.155578 8454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:48.156852 8435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:48.157052 8435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "6c4da626004c4351b5a043a74ec3fb28"
format_stamp: "Formatted at 2026-08-08 11:48:48 on dist-test-slave-39l8"
server_key: "b55afc781f93ca9a439ff53d91dcb610"
server_key_iv: "44b303454be161b756de51d8e0b89904"
server_key_version: "encryptionkey@0"
I20260808 11:48:48.157260 8435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:48.173693 8435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:48.177713 8435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:48.178012 8435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:48.179020 8435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:48.180362 8435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:48.180474 8435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:48.180641 8435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:48.180718 8435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:48.183807 8232 heartbeater.cc:503] Master 127.13.158.126:45271 was elected leader, sending a full tablet report...
I20260808 11:48:48.215555 8435 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:37537
I20260808 11:48:48.215605 8625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:37537 every 8 connection(s)
I20260808 11:48:48.216861 8435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:48:48.219062 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 8435
I20260808 11:48:48.219264 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:48:48.219889 13945 external_mini_cluster.cc:1428] Setting key 9f70d65235b9e0b069b5df17bbf69c3a
Aug 08 11:48:48 dist-test-slave-39l8 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189728, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:48.236948 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.219473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:54693 (local address 127.13.158.126:45271)
0808 11:48:48.219963 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.219970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.221039 (+ 1069us) server_negotiation.cc:407] Connection header received
0808 11:48:48.223103 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.223108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.223174 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.223263 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:48.224667 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.225410 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.226979 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.227367 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.227460 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.230810 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:48.230815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:48.230818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:48.230851 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:48.233980 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.234826 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.234831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.234834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.234953 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.235522 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.235533 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.235535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.235789 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:48.235908 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.236544 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.236696 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":234,"threads_started":1}
I20260808 11:48:48.237843 8632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.220074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:45271 (local address 127.13.158.67:54693)
0808 11:48:48.220871 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:48.220929 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:48.222850 (+ 1921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:48.223482 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:48.223489 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:48.223956 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:48.224510 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.224526 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.225548 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.225552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.226856 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.226868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.227555 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.227560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.227749 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.228366 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:48.228388 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:48.230504 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:48.234122 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:48.234132 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:48.234147 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:48.234669 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:48.235139 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:48.235145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:48.235148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:48.235400 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:48.235903 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:48.235909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:48.236191 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 11:48:48.236430 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 11:48:48.236769 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":339,"threads_started":1}
I20260808 11:48:48.239207 8628 heartbeater.cc:347] Connected to a master server at 127.13.158.126:45271
I20260808 11:48:48.239500 8628 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:48.240228 8628 heartbeater.cc:511] Master 127.13.158.126:45271 requested a full tablet report, sending...
I20260808 11:48:48.241608 7955 ts_manager.cc:194] Registered new tserver with Master: 6c4da626004c4351b5a043a74ec3fb28 (127.13.158.67:37537)
I20260808 11:48:48.242403 7955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:54693
I20260808 11:48:48.247380 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:48:48.264633 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.251431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.13.158.126:45271)
0808 11:48:48.251657 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.251666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.252039 (+ 373us) server_negotiation.cc:407] Connection header received
0808 11:48:48.252212 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.252217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.252279 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.252366 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:48.253765 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.254806 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.256312 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.256702 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.256797 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.259488 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:48.259493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:48.259495 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:48.259525 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:48.262114 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.262872 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.262876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.262878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.262986 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.263356 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.263359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.263361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.263640 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:48.263836 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.264130 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.264296 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 11:48:48.268388 7955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:48:48.275506 7955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:48:48.322600 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.289325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45320 (local address 127.13.158.65:41411)
0808 11:48:48.313098 (+ 23773us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.313110 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.313892 (+ 782us) server_negotiation.cc:407] Connection header received
0808 11:48:48.314026 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.314033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.314262 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.314472 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:48.315501 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.317133 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.320095 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.321235 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.321443 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.321742 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.322030 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.322185 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23578,"thread_start_us":23404,"threads_started":1}
I20260808 11:48:48.322646 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.288547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:41411 (local address 127.0.0.1:45320)
0808 11:48:48.313566 (+ 25019us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:48.313586 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:48.313740 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:48.314507 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:48.314514 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:48.314561 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:48.315308 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.315321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.317342 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.317346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.319892 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.319907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.321432 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.321438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.321612 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.321735 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 11:48:48.321968 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 11:48:48.322157 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24888,"mutex_wait_us":360,"spinlock_wait_cycles":6272,"thread_start_us":24753,"threads_started":1}
I20260808 11:48:48.323094 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.287669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:37537 (local address 127.0.0.1:34758)
0808 11:48:48.312214 (+ 24545us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:48.312270 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:48.312498 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:48.314628 (+ 2130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:48.314633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:48.314665 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:48.315413 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.315425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.317286 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.317292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.319602 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.319615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.322052 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.322060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.322351 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.322423 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 11:48:48.322511 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 11:48:48.322645 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24146,"mutex_wait_us":276,"thread_start_us":25681,"threads_started":1}
I20260808 11:48:48.323092 8654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.288088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50388 (local address 127.13.158.66:35133)
0808 11:48:48.311302 (+ 23214us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.311312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.313329 (+ 2017us) server_negotiation.cc:407] Connection header received
0808 11:48:48.313858 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.313869 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.314103 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.314239 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:48.315604 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.316830 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.320143 (+ 3313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.321540 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.321675 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.321851 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.322597 (+ 746us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.322810 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22993,"thread_start_us":23234,"threads_started":1}
I20260808 11:48:48.326277 8130 tablet_service.cc:1467] Processing CreateTablet for tablet b182583e7a534de0be81b9fb211c7f0f (DEFAULT_TABLE table=test-table [id=fcfe521a9d0c42f48e148fd83a115a32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:48.326992 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.287882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34758 (local address 127.13.158.67:37537)
0808 11:48:48.313998 (+ 26116us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.314006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.314034 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:48:48.314134 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.314140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.314317 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.314491 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:48.315587 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.317139 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.319774 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.321383 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.321511 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.321812 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.323550 (+ 1738us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.323717 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25933,"thread_start_us":25768,"threads_started":1}
I20260808 11:48:48.323881 8651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.288002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:35133 (local address 127.0.0.1:50388)
0808 11:48:48.313160 (+ 25158us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:48.313187 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:48.313669 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:48.314507 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:48.314514 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:48.314554 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:48.315294 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.315315 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.316968 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.316976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.319895 (+ 2919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.319907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.322068 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.322075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.322356 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.322424 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 11:48:48.323355 (+ 931us) client_negotiation.cc:240] Negotiation successful
0808 11:48:48.323510 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24967,"mutex_wait_us":394,"spinlock_wait_cycles":1280,"thread_start_us":23659,"threads_started":1}
I20260808 11:48:48.328099 8130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b182583e7a534de0be81b9fb211c7f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:48.330835 8331 tablet_service.cc:1467] Processing CreateTablet for tablet b182583e7a534de0be81b9fb211c7f0f (DEFAULT_TABLE table=test-table [id=fcfe521a9d0c42f48e148fd83a115a32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:48.331199 8525 tablet_service.cc:1467] Processing CreateTablet for tablet b182583e7a534de0be81b9fb211c7f0f (DEFAULT_TABLE table=test-table [id=fcfe521a9d0c42f48e148fd83a115a32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:48.332319 8331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b182583e7a534de0be81b9fb211c7f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:48.332612 8525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b182583e7a534de0be81b9fb211c7f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:48.339504 8657 tablet_bootstrap.cc:492] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Bootstrap starting.
I20260808 11:48:48.342772 8657 tablet_bootstrap.cc:654] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:48.343815 8657 log.cc:824] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:48.346517 8657 tablet_bootstrap.cc:492] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: No bootstrap required, opened a new log
I20260808 11:48:48.346961 8657 ts_tablet_manager.cc:1397] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260808 11:48:48.347172 8661 tablet_bootstrap.cc:492] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Bootstrap starting.
I20260808 11:48:48.349045 8660 tablet_bootstrap.cc:492] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Bootstrap starting.
I20260808 11:48:48.350736 8661 tablet_bootstrap.cc:654] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:48.352062 8661 log.cc:824] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:48.352424 8660 tablet_bootstrap.cc:654] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:48.353338 8660 log.cc:824] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:48.356192 8661 tablet_bootstrap.cc:492] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: No bootstrap required, opened a new log
I20260808 11:48:48.356477 8660 tablet_bootstrap.cc:492] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: No bootstrap required, opened a new log
I20260808 11:48:48.356491 8661 ts_tablet_manager.cc:1397] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260808 11:48:48.356732 8660 ts_tablet_manager.cc:1397] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260808 11:48:48.359091 8657 raft_consensus.cc:348] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.359505 8657 raft_consensus.cc:374] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:48.359663 8657 raft_consensus.cc:729] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242f137c49cd4220853c7bbe60e9b933, State: Initialized, Role: FOLLOWER
I20260808 11:48:48.360288 8657 consensus_queue.cc:260] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.362284 8657 ts_tablet_manager.cc:1428] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260808 11:48:48.365690 8661 raft_consensus.cc:348] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.365716 8660 raft_consensus.cc:348] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.366097 8661 raft_consensus.cc:374] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:48.366122 8660 raft_consensus.cc:374] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:48.366195 8661 raft_consensus.cc:729] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c263c0b65b44f6b3ad216f50e97c15, State: Initialized, Role: FOLLOWER
I20260808 11:48:48.366235 8660 raft_consensus.cc:729] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c4da626004c4351b5a043a74ec3fb28, State: Initialized, Role: FOLLOWER
I20260808 11:48:48.366917 8660 consensus_queue.cc:260] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.367278 8661 consensus_queue.cc:260] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.368808 8628 heartbeater.cc:503] Master 127.13.158.126:45271 was elected leader, sending a full tablet report...
I20260808 11:48:48.369150 8427 heartbeater.cc:503] Master 127.13.158.126:45271 was elected leader, sending a full tablet report...
I20260808 11:48:48.369467 8660 ts_tablet_manager.cc:1428] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
I20260808 11:48:48.369905 8661 ts_tablet_manager.cc:1428] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260808 11:48:48.386469 8667 raft_consensus.cc:482] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:48.386861 8667 raft_consensus.cc:504] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.388891 8667 leader_election.cc:288] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c263c0b65b44f6b3ad216f50e97c15 (127.13.158.66:35133), 242f137c49cd4220853c7bbe60e9b933 (127.13.158.65:41411)
I20260808 11:48:48.395412 8654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.389260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:52125 (local address 127.13.158.66:35133)
0808 11:48:48.389480 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.389486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.389506 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:48:48.389655 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.389660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.389733 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.389814 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:48.390337 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.391381 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.393332 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.394576 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.394683 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.394804 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.395123 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.395220 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 11:48:48.395499 8632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.389040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:35133 (local address 127.13.158.67:52125)
0808 11:48:48.389350 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:48.389372 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:48.389486 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:48.389999 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:48.390003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:48.390027 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:48.390174 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.390184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.391536 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.391543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.393156 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.393169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.394757 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.394764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.394957 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.394995 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 11:48:48.395094 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 11:48:48.395233 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260808 11:48:48.396626 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.389169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:41411 (local address 127.13.158.67:49599)
0808 11:48:48.390199 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:48.390214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:48.390361 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:48.390677 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:48.390682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:48.390741 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:48.391069 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.391081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.392530 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.392534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.394398 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:48.394413 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.396042 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.396048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:48.396214 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.396238 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:48:48.396317 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 11:48:48.396397 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"thread_start_us":691,"threads_started":1}
I20260808 11:48:48.396771 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.389737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:49599 (local address 127.13.158.65:41411)
0808 11:48:48.389935 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.389941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.390347 (+ 406us) server_negotiation.cc:407] Connection header received
0808 11:48:48.390492 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.390495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.390549 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.390616 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:48.391218 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.392372 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.394566 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.395854 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.395967 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.396015 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.396390 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.396522 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:48:48.396624 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b182583e7a534de0be81b9fb211c7f0f" candidate_uuid: "6c4da626004c4351b5a043a74ec3fb28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c263c0b65b44f6b3ad216f50e97c15" is_pre_election: true
I20260808 11:48:48.397197 8363 raft_consensus.cc:2454] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c4da626004c4351b5a043a74ec3fb28 in term 0.
I20260808 11:48:48.397938 8456 leader_election.cc:302] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c263c0b65b44f6b3ad216f50e97c15, 6c4da626004c4351b5a043a74ec3fb28; no voters:
I20260808 11:48:48.397810 8158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b182583e7a534de0be81b9fb211c7f0f" candidate_uuid: "6c4da626004c4351b5a043a74ec3fb28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242f137c49cd4220853c7bbe60e9b933" is_pre_election: true
I20260808 11:48:48.398382 8667 raft_consensus.cc:2781] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:48:48.398391 8158 raft_consensus.cc:2454] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c4da626004c4351b5a043a74ec3fb28 in term 0.
I20260808 11:48:48.398537 8667 raft_consensus.cc:482] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:48:48.398667 8667 raft_consensus.cc:3037] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:48.401000 8667 raft_consensus.cc:504] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.401594 8667 leader_election.cc:288] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [CANDIDATE]: Term 1 election: Requested vote from peers 13c263c0b65b44f6b3ad216f50e97c15 (127.13.158.66:35133), 242f137c49cd4220853c7bbe60e9b933 (127.13.158.65:41411)
I20260808 11:48:48.401957 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b182583e7a534de0be81b9fb211c7f0f" candidate_uuid: "6c4da626004c4351b5a043a74ec3fb28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c263c0b65b44f6b3ad216f50e97c15"
I20260808 11:48:48.402163 8363 raft_consensus.cc:3037] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 0 FOLLOWER]: Advancing to term 1
W20260808 11:48:48.402171 8233 tablet.cc:2369] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:48.402099 8158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b182583e7a534de0be81b9fb211c7f0f" candidate_uuid: "6c4da626004c4351b5a043a74ec3fb28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242f137c49cd4220853c7bbe60e9b933"
I20260808 11:48:48.402398 8158 raft_consensus.cc:3037] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:48.404507 8363 raft_consensus.cc:2454] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c4da626004c4351b5a043a74ec3fb28 in term 1.
I20260808 11:48:48.404764 8158 raft_consensus.cc:2454] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c4da626004c4351b5a043a74ec3fb28 in term 1.
I20260808 11:48:48.404906 8456 leader_election.cc:302] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c263c0b65b44f6b3ad216f50e97c15, 6c4da626004c4351b5a043a74ec3fb28; no voters:
I20260808 11:48:48.405251 8667 raft_consensus.cc:2781] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:48.405679 8667 raft_consensus.cc:686] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 1 LEADER]: Becoming Leader. State: Replica: 6c4da626004c4351b5a043a74ec3fb28, State: Running, Role: LEADER
I20260808 11:48:48.406174 8667 consensus_queue.cc:237] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } }
I20260808 11:48:48.411284 7955 catalog_manager.cc:5495] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c4da626004c4351b5a043a74ec3fb28 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "6c4da626004c4351b5a043a74ec3fb28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c4da626004c4351b5a043a74ec3fb28" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 37537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 } health_report { overall_health: UNKNOWN } } }
W20260808 11:48:48.435333 8428 tablet.cc:2369] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:48.453845 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.448556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34760 (local address 127.13.158.67:37537)
0808 11:48:48.448782 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.448787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.448810 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:48:48.449021 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.449025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.449101 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.449232 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:48:48.449613 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.450372 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.451750 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.452152 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.452247 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.452721 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:48:48.453203 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:48:48.453337 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.453474 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.453555 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:48:48.462203 8158 raft_consensus.cc:1261] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 1 FOLLOWER]: Refusing update from remote peer 6c4da626004c4351b5a043a74ec3fb28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:48.462200 8363 raft_consensus.cc:1261] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 1 FOLLOWER]: Refusing update from remote peer 6c4da626004c4351b5a043a74ec3fb28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:48.463130 8673 consensus_queue.cc:1035] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c263c0b65b44f6b3ad216f50e97c15" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 35133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:48.463526 8667 consensus_queue.cc:1035] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242f137c49cd4220853c7bbe60e9b933" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 41411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:48:48.468421 8629 tablet.cc:2369] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:48.479943 8678 mvcc.cc:205] Tried to move back new op lower bound from 7316233127770427392 to 7316233127556276224. Current Snapshot: MvccSnapshot[applied={T|T < 7316233127770427392}]
I20260808 11:48:48.502039 7952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:48.502382 7952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:48.506131 7952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dddf567a91f44c8db1ce67049d68a09e: Sending DeleteTablet for 3 replicas of tablet b182583e7a534de0be81b9fb211c7f0f
I20260808 11:48:48.507277 8525 tablet_service.cc:1514] Processing DeleteTablet for tablet b182583e7a534de0be81b9fb211c7f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:48 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:34758
I20260808 11:48:48.507441 8331 tablet_service.cc:1514] Processing DeleteTablet for tablet b182583e7a534de0be81b9fb211c7f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:48 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:50388
I20260808 11:48:48.507441 8130 tablet_service.cc:1514] Processing DeleteTablet for tablet b182583e7a534de0be81b9fb211c7f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:48 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:45320
I20260808 11:48:48.509197 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 8042
I20260808 11:48:48.521853 8690 tablet_replica.cc:330] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: stopping tablet replica
I20260808 11:48:48.522972 8693 tablet_replica.cc:330] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: stopping tablet replica
I20260808 11:48:48.523056 8690 raft_consensus.cc:2229] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:48:48.523983 8690 raft_consensus.cc:2258] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:48.524006 8693 raft_consensus.cc:2229] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:48.524487 8693 raft_consensus.cc:2258] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:48.526973 8690 ts_tablet_manager.cc:1905] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:48.527697 8694 tablet_replica.cc:330] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: stopping tablet replica
I20260808 11:48:48.527802 8693 ts_tablet_manager.cc:1905] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:48.528168 8694 raft_consensus.cc:2229] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:48.528805 8694 raft_consensus.cc:2258] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:48.532799 8694 ts_tablet_manager.cc:1905] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:48.534968 8693 ts_tablet_manager.cc:1918] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:48.535184 8693 log.cc:1196] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/b182583e7a534de0be81b9fb211c7f0f
I20260808 11:48:48.535825 8693 ts_tablet_manager.cc:1939] T b182583e7a534de0be81b9fb211c7f0f P 13c263c0b65b44f6b3ad216f50e97c15: Deleting consensus metadata
I20260808 11:48:48.538086 7930 catalog_manager.cc:4841] TS 13c263c0b65b44f6b3ad216f50e97c15 (127.13.158.66:35133): tablet b182583e7a534de0be81b9fb211c7f0f (table test-table [id=fcfe521a9d0c42f48e148fd83a115a32]) successfully deleted
I20260808 11:48:48.539696 8694 ts_tablet_manager.cc:1918] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:48.539938 8694 log.cc:1196] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/b182583e7a534de0be81b9fb211c7f0f
I20260808 11:48:48.540508 8694 ts_tablet_manager.cc:1939] T b182583e7a534de0be81b9fb211c7f0f P 242f137c49cd4220853c7bbe60e9b933: Deleting consensus metadata
I20260808 11:48:48.541610 8690 ts_tablet_manager.cc:1918] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 08 11:48:48 dist-test-slave-39l8 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:48:48.541817 8690 log.cc:1196] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/b182583e7a534de0be81b9fb211c7f0f
I20260808 11:48:48.542286 7928 catalog_manager.cc:4841] TS 242f137c49cd4220853c7bbe60e9b933 (127.13.158.65:41411): tablet b182583e7a534de0be81b9fb211c7f0f (table test-table [id=fcfe521a9d0c42f48e148fd83a115a32]) successfully deleted
I20260808 11:48:48.542441 8690 ts_tablet_manager.cc:1939] T b182583e7a534de0be81b9fb211c7f0f P 6c4da626004c4351b5a043a74ec3fb28: Deleting consensus metadata
I20260808 11:48:48.544080 7934 catalog_manager.cc:4841] TS 6c4da626004c4351b5a043a74ec3fb28 (127.13.158.67:37537): tablet b182583e7a534de0be81b9fb211c7f0f (table test-table [id=fcfe521a9d0c42f48e148fd83a115a32]) successfully deleted
I20260808 11:48:48.548928 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.510639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.13.158.65:41411)
0808 11:48:48.510843 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.510851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.528825 (+ 17974us) server_negotiation.cc:407] Connection header received
0808 11:48:48.529328 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.529335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.529431 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.529513 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:48.531342 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.532531 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.534414 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.535101 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.535220 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.542960 (+ 7740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:48.542975 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:48.542992 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:48.543028 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:48.546088 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.546992 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.546998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.547000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.547127 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.547555 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.547558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.547560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.547891 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:48.548103 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.548457 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.548608 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 11:48:48.549541 8223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:48.673990 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 8042
I20260808 11:48:48.702518 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 8240
Aug 08 11:48:48 dist-test-slave-39l8 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:48:48.720916 8654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.703423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50396 (local address 127.13.158.66:35133)
0808 11:48:48.703633 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.703640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.703753 (+ 113us) server_negotiation.cc:407] Connection header received
0808 11:48:48.703924 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.703929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.704003 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.704107 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:48.705784 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.706974 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.708972 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.709632 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.709747 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.714981 (+ 5234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:48.714992 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:48.715007 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:48.715041 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:48.718050 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.718755 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.718771 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.718773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.718901 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.719267 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.719272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.719275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.719666 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:48.719960 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.720286 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.720500 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:48:48.721674 8417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:48.809381 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 8240
I20260808 11:48:48.845320 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 8435
Aug 08 11:48:48 dist-test-slave-39l8 krb5kdc[7890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:48:48.861992 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:48.846245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.13.158.67:37537)
0808 11:48:48.846445 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:48.846451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:48.846471 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:48:48.846626 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:48.846630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:48.846747 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:48.846891 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:48.848231 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.849074 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.850348 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:48.850865 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:48.851004 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:48.855602 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:48.855614 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:48.855629 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:48.855668 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:48.858798 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.859491 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.859499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.859503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.859673 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:48.860108 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:48.860111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:48.860113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:48.860512 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:48.860850 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:48.861238 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 11:48:48.861471 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:48:48.862880 8621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:48.981671 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 8435
I20260808 11:48:49.009104 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 7905
I20260808 11:48:49.010210 7992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:49.121104 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 7905
2026-08-08T11:48:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4727 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T11:48:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:48:49Z Disabled control of system clock
I20260808 11:48:49.234241 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:35877
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:35877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:48:49.570921 8708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:49.571199 8708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:49.571282 8708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:49.578200 8708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:48:49.578299 8708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:49.578337 8708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:49.578387 8708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:48:49.578431 8708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:48:49.587209 8708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:35877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:35877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:49.588419 8708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:49.590303 8708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:49.601838 8723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:49.601780 8718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:49.601769 8720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:49.602854 8708 server_base.cc:1034] running on GCE node
I20260808 11:48:49.603571 8708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:49.604888 8708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:49.606124 8708 hybrid_clock.cc:648] HybridClock initialized: now 1786189729606032 us; error 83 us; skew 500 ppm
I20260808 11:48:49.606472 8708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:49.608788 8708 webserver.cc:460] Webserver started at http://127.13.158.126:39745/ using document root <none> and password file <none>
I20260808 11:48:49.609455 8708 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:49.609596 8708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:49.609923 8708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:49.612795 8708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "b3476238d275486aa76bcb204d45b874"
format_stamp: "Formatted at 2026-08-08 11:48:49 on dist-test-slave-39l8"
server_key: "94666396ba9e0423d1213b862381a3e3"
server_key_iv: "1393f0e747294e829e9b55bf4a1aa56c"
server_key_version: "encryptionkey@0"
I20260808 11:48:49.613603 8708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "b3476238d275486aa76bcb204d45b874"
format_stamp: "Formatted at 2026-08-08 11:48:49 on dist-test-slave-39l8"
server_key: "94666396ba9e0423d1213b862381a3e3"
server_key_iv: "1393f0e747294e829e9b55bf4a1aa56c"
server_key_version: "encryptionkey@0"
I20260808 11:48:49.618469 8708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 11:48:49.621814 8733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:49.623221 8708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:48:49.623414 8708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "b3476238d275486aa76bcb204d45b874"
format_stamp: "Formatted at 2026-08-08 11:48:49 on dist-test-slave-39l8"
server_key: "94666396ba9e0423d1213b862381a3e3"
server_key_iv: "1393f0e747294e829e9b55bf4a1aa56c"
server_key_version: "encryptionkey@0"
I20260808 11:48:49.623626 8708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:49.635183 8708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:49.635939 8708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:49.636229 8708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:49.654660 8708 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:35877
I20260808 11:48:49.654680 8808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:35877 every 8 connection(s)
I20260808 11:48:49.655964 8708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:48:49.659567 8809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:49.661580 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 8708
I20260808 11:48:49.661777 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:48:49.662355 13945 external_mini_cluster.cc:1428] Setting key be4c49bc90b42e09fb0b11ac09ab89c9
I20260808 11:48:49.672173 8809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874: Bootstrap starting.
I20260808 11:48:49.675230 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:49.664265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.13.158.126:35877)
0808 11:48:49.665276 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:49.665286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:49.665335 (+ 49us) server_negotiation.cc:407] Connection header received
0808 11:48:49.666903 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:49.666926 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:49.667357 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:49.667724 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:49.668246 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:49.669418 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:49.670793 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:49.671440 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:49.671651 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:49.672937 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:49.672945 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:49.672957 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:49.673472 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:49.673559 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:49.673579 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:49.673734 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:49.673960 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 11:48:49.674255 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":350,"threads_started":1}
I20260808 11:48:49.675397 8809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:49.676532 8809 log.cc:824] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:49.679354 8809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874: No bootstrap required, opened a new log
I20260808 11:48:49.686190 8809 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3476238d275486aa76bcb204d45b874" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35877 } }
I20260808 11:48:49.686574 8809 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:49.686687 8809 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3476238d275486aa76bcb204d45b874, State: Initialized, Role: FOLLOWER
I20260808 11:48:49.687340 8809 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3476238d275486aa76bcb204d45b874" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35877 } }
I20260808 11:48:49.687585 8809 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:48:49.687738 8809 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:48:49.687953 8809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:49.689997 8809 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3476238d275486aa76bcb204d45b874" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35877 } }
I20260808 11:48:49.690508 8809 leader_election.cc:302] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3476238d275486aa76bcb204d45b874; no voters:
I20260808 11:48:49.691223 8809 leader_election.cc:288] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:48:49.691327 8815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:49.692082 8815 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [term 1 LEADER]: Becoming Leader. State: Replica: b3476238d275486aa76bcb204d45b874, State: Running, Role: LEADER
I20260808 11:48:49.692648 8815 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3476238d275486aa76bcb204d45b874" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35877 } }
I20260808 11:48:49.693437 8809 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:48:49.695540 8816 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3476238d275486aa76bcb204d45b874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3476238d275486aa76bcb204d45b874" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35877 } } }
I20260808 11:48:49.695904 8816 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:49.696779 8829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:48:49.696590 8820 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3476238d275486aa76bcb204d45b874. Latest consensus state: current_term: 1 leader_uuid: "b3476238d275486aa76bcb204d45b874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3476238d275486aa76bcb204d45b874" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 35877 } } }
I20260808 11:48:49.697763 8820 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874 [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:49.706439 8829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:48:49.714721 8829 catalog_manager.cc:1347] Generated new cluster ID: d11dcc7b395544b89cb7e0211121ad70
I20260808 11:48:49.714824 8829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:48:49.726507 8829 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:48:49.728027 8829 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:48:49.736199 8829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874: Generated new TSK 0
I20260808 11:48:49.736958 8829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:48:49.745708 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35877
--builtin_ntp_servers=127.13.158.84:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:48:50.093530 8845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:50.093818 8845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:50.093986 8845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:50.101557 8845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:50.101655 8845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:50.101841 8845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:48:50.111145 8845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:50.112516 8845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:50.114287 8845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:50.127858 8851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:50.127833 8852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:50.127772 8854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:50.128665 8845 server_base.cc:1034] running on GCE node
I20260808 11:48:50.129261 8845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:50.130201 8845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:50.131413 8845 hybrid_clock.cc:648] HybridClock initialized: now 1786189730131369 us; error 35 us; skew 500 ppm
I20260808 11:48:50.131737 8845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:50.133988 8845 webserver.cc:460] Webserver started at http://127.13.158.65:33861/ using document root <none> and password file <none>
I20260808 11:48:50.134773 8845 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:50.134900 8845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:50.135264 8845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:50.138224 8845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "812fee04617646d3af2db01252b5149c"
format_stamp: "Formatted at 2026-08-08 11:48:50 on dist-test-slave-39l8"
server_key: "ceed8ac785999afaed2f6cb79ba119d3"
server_key_iv: "80c0b383ed5ba1b721c7da0afccc804d"
server_key_version: "encryptionkey@0"
I20260808 11:48:50.139143 8845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "812fee04617646d3af2db01252b5149c"
format_stamp: "Formatted at 2026-08-08 11:48:50 on dist-test-slave-39l8"
server_key: "ceed8ac785999afaed2f6cb79ba119d3"
server_key_iv: "80c0b383ed5ba1b721c7da0afccc804d"
server_key_version: "encryptionkey@0"
I20260808 11:48:50.144317 8845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260808 11:48:50.147786 8867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:50.149114 8845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:48:50.149312 8845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "812fee04617646d3af2db01252b5149c"
format_stamp: "Formatted at 2026-08-08 11:48:50 on dist-test-slave-39l8"
server_key: "ceed8ac785999afaed2f6cb79ba119d3"
server_key_iv: "80c0b383ed5ba1b721c7da0afccc804d"
server_key_version: "encryptionkey@0"
I20260808 11:48:50.149544 8845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:50.174090 8845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:50.175040 8845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:50.175374 8845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:50.176357 8845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:50.177846 8845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:50.177999 8845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:50.178144 8845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:50.178241 8845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:50.212513 8845 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:37397
I20260808 11:48:50.212574 9051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:37397 every 8 connection(s)
I20260808 11:48:50.213892 8845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:48:50.219751 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 8845
I20260808 11:48:50.219939 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:48:50.220410 13945 external_mini_cluster.cc:1428] Setting key e4c7a0edafb3b0d0c705469db18b33f9
I20260808 11:48:50.223281 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35877
--builtin_ntp_servers=127.13.158.84:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:48:50.227237 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:50.216643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:55963 (local address 127.13.158.126:35877)
0808 11:48:50.217114 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:50.217119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:50.218311 (+ 1192us) server_negotiation.cc:407] Connection header received
0808 11:48:50.220408 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:50.220414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:50.220531 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:50.220683 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:50.221753 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:50.222549 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:50.224278 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:50.224690 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:50.224832 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:50.226291 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:50.226301 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:50.226304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:50.226323 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:50.226366 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:50.226378 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:50.226482 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:50.226757 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 11:48:50.226878 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":233,"threads_started":1}
I20260808 11:48:50.228297 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:50.217257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:35877 (local address 127.13.158.65:55963)
0808 11:48:50.218129 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:50.218177 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:50.220114 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:50.220909 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:50.220918 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:50.221011 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:50.221575 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:50.221592 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:50.222772 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:50.222776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:50.224103 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:50.224116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:50.224946 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:50.224953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:50.225195 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:50.225938 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:50.225941 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:50.225982 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:50.226069 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:50.226484 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:50.226497 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:50.226508 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 11:48:50.226715 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 11:48:50.227199 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":276,"threads_started":1}
I20260808 11:48:50.230031 9052 heartbeater.cc:347] Connected to a master server at 127.13.158.126:35877
I20260808 11:48:50.230369 9052 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:50.231156 9052 heartbeater.cc:511] Master 127.13.158.126:35877 requested a full tablet report, sending...
I20260808 11:48:50.233493 8759 ts_manager.cc:194] Registered new tserver with Master: 812fee04617646d3af2db01252b5149c (127.13.158.65:37397)
I20260808 11:48:50.235744 8759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.65:55963
W20260808 11:48:50.559324 9059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:50.559631 9059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:50.559808 9059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:50.566881 9059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:50.567019 9059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:50.567203 9059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:48:50.575902 9059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:50.577175 9059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:50.578910 9059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:50.591925 9067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:50.591888 9070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:50.591988 9072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:50.593228 9059 server_base.cc:1034] running on GCE node
I20260808 11:48:50.593921 9059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:50.594928 9059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:50.596158 9059 hybrid_clock.cc:648] HybridClock initialized: now 1786189730596087 us; error 57 us; skew 500 ppm
I20260808 11:48:50.596477 9059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:50.599021 9059 webserver.cc:460] Webserver started at http://127.13.158.66:38815/ using document root <none> and password file <none>
I20260808 11:48:50.599704 9059 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:50.599838 9059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:50.600186 9059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:50.603018 9059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "3b518a23f3754db7bc0be970076c6e4e"
format_stamp: "Formatted at 2026-08-08 11:48:50 on dist-test-slave-39l8"
server_key: "e4c3fdbdfd4ab3abfd59b4279f77bbbb"
server_key_iv: "490270e59357180ec519db17032027a7"
server_key_version: "encryptionkey@0"
I20260808 11:48:50.603833 9059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "3b518a23f3754db7bc0be970076c6e4e"
format_stamp: "Formatted at 2026-08-08 11:48:50 on dist-test-slave-39l8"
server_key: "e4c3fdbdfd4ab3abfd59b4279f77bbbb"
server_key_iv: "490270e59357180ec519db17032027a7"
server_key_version: "encryptionkey@0"
I20260808 11:48:50.609375 9059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:48:50.613080 9081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:50.614328 9059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:48:50.614533 9059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "3b518a23f3754db7bc0be970076c6e4e"
format_stamp: "Formatted at 2026-08-08 11:48:50 on dist-test-slave-39l8"
server_key: "e4c3fdbdfd4ab3abfd59b4279f77bbbb"
server_key_iv: "490270e59357180ec519db17032027a7"
server_key_version: "encryptionkey@0"
I20260808 11:48:50.614784 9059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:50.648010 9059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:50.648857 9059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:50.649175 9059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:50.650121 9059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:50.651619 9059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:50.651793 9059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:50.651932 9059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:50.652045 9059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:50.687472 9059 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:37403
I20260808 11:48:50.687515 9259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:37403 every 8 connection(s)
I20260808 11:48:50.688809 9059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:48:50.697899 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 9059
I20260808 11:48:50.698079 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:48:50.698523 13945 external_mini_cluster.cc:1428] Setting key cee9d797d7609981d7739e0db55d9191
I20260808 11:48:50.701236 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:50.691431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:54219 (local address 127.13.158.126:35877)
0808 11:48:50.691657 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:50.691663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:50.693018 (+ 1355us) server_negotiation.cc:407] Connection header received
0808 11:48:50.694777 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:50.694782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:50.694858 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:50.694942 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:50.695932 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:50.696633 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:50.698142 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:50.698672 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:50.698838 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:50.700426 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:50.700432 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:50.700437 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:50.700463 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:50.700494 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:50.700506 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:50.700619 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:50.700848 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 11:48:50.700941 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:48:50.701514 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35877
--builtin_ntp_servers=127.13.158.84:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:48:50.702309 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:50.691974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:35877 (local address 127.13.158.66:54219)
0808 11:48:50.692857 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:50.692890 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:50.694516 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:50.695163 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:50.695170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:50.695257 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:50.695784 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:50.695799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:50.696774 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:50.696778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:50.697945 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:50.697997 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:50.698929 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:50.698936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:50.699276 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:50.700090 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:50.700099 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:50.700151 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:50.700237 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:50.700621 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:50.700634 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:50.700643 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:48:50.700836 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 11:48:50.701189 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":282,"threads_started":1}
I20260808 11:48:50.703662 9260 heartbeater.cc:347] Connected to a master server at 127.13.158.126:35877
I20260808 11:48:50.703995 9260 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:50.705092 9260 heartbeater.cc:511] Master 127.13.158.126:35877 requested a full tablet report, sending...
I20260808 11:48:50.706835 8759 ts_manager.cc:194] Registered new tserver with Master: 3b518a23f3754db7bc0be970076c6e4e (127.13.158.66:37403)
I20260808 11:48:50.707842 8759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.66:54219
W20260808 11:48:51.038508 9264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:51.038770 9264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:51.038851 9264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:51.045820 9264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:51.045912 9264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:51.046013 9264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:48:51.054877 9264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:35877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:51.056032 9264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:51.057693 9264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:51.070897 9271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:51.070847 9273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:51.070935 9277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:51.072237 9264 server_base.cc:1034] running on GCE node
I20260808 11:48:51.072904 9264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:51.073751 9264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:51.074993 9264 hybrid_clock.cc:648] HybridClock initialized: now 1786189731074959 us; error 44 us; skew 500 ppm
I20260808 11:48:51.075312 9264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:51.077553 9264 webserver.cc:460] Webserver started at http://127.13.158.67:39977/ using document root <none> and password file <none>
I20260808 11:48:51.078244 9264 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:51.078387 9264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:51.078751 9264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:51.081579 9264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "022d24dc08aa4349b9c7f64fd6989bf8"
format_stamp: "Formatted at 2026-08-08 11:48:51 on dist-test-slave-39l8"
server_key: "75dbda5f1a6eeddc800e9d3402b60f1d"
server_key_iv: "1a22dcef3aeb022a3a028c405c707692"
server_key_version: "encryptionkey@0"
I20260808 11:48:51.082389 9264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "022d24dc08aa4349b9c7f64fd6989bf8"
format_stamp: "Formatted at 2026-08-08 11:48:51 on dist-test-slave-39l8"
server_key: "75dbda5f1a6eeddc800e9d3402b60f1d"
server_key_iv: "1a22dcef3aeb022a3a028c405c707692"
server_key_version: "encryptionkey@0"
I20260808 11:48:51.087507 9264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 11:48:51.091063 9288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:51.092404 9264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:48:51.092607 9264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "022d24dc08aa4349b9c7f64fd6989bf8"
format_stamp: "Formatted at 2026-08-08 11:48:51 on dist-test-slave-39l8"
server_key: "75dbda5f1a6eeddc800e9d3402b60f1d"
server_key_iv: "1a22dcef3aeb022a3a028c405c707692"
server_key_version: "encryptionkey@0"
I20260808 11:48:51.092841 9264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:51.130836 9264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:51.131721 9264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:51.132076 9264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:51.133188 9264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:51.134655 9264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:51.134816 9264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 11:48:51.134948 9264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:51.135073 9264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:51.168686 9264 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:43577
I20260808 11:48:51.168746 9449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:43577 every 8 connection(s)
I20260808 11:48:51.169991 9264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:48:51.173969 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 9264
I20260808 11:48:51.174140 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:48:51.174741 13945 external_mini_cluster.cc:1428] Setting key 5ff1f0753044c7f6aa24b71e289c2537
I20260808 11:48:51.182911 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.172626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:41381 (local address 127.13.158.126:35877)
0808 11:48:51.172812 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.172817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.174283 (+ 1466us) server_negotiation.cc:407] Connection header received
0808 11:48:51.176610 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.176616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.176715 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.176805 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:51.178049 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.178879 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.180351 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.180762 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.180853 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.182153 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:51.182157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:51.182160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:51.182178 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:51.182207 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:51.182218 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:51.182286 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.182505 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.182580 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:48:51.183830 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.173043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:35877 (local address 127.13.158.67:41381)
0808 11:48:51.174088 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.174137 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:51.176263 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:51.177064 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:51.177072 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:51.177153 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:51.177876 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.177901 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.179026 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.179030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.180227 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.180238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.180949 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.180953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.181145 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.181868 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:51.181870 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:51.181907 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:51.181983 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:51.182310 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:51.182318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:51.182323 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 11:48:51.182460 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 11:48:51.182824 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":281,"threads_started":1}
I20260808 11:48:51.185132 9450 heartbeater.cc:347] Connected to a master server at 127.13.158.126:35877
I20260808 11:48:51.185451 9450 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:51.186180 9450 heartbeater.cc:511] Master 127.13.158.126:35877 requested a full tablet report, sending...
I20260808 11:48:51.187618 8759 ts_manager.cc:194] Registered new tserver with Master: 022d24dc08aa4349b9c7f64fd6989bf8 (127.13.158.67:43577)
I20260808 11:48:51.188514 8759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.67:41381
I20260808 11:48:51.190184 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:48:51.200871 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.194619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.13.158.126:35877)
0808 11:48:51.194918 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.194927 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.195328 (+ 401us) server_negotiation.cc:407] Connection header received
0808 11:48:51.195532 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.195539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.195619 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.195735 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:51.196227 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.196904 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.198304 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.198767 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.198907 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.200156 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:51.200163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:51.200173 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:51.200190 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:51.200226 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:51.200238 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:51.200388 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.200575 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.200630 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:48:51.204721 8759 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:48:51.211449 8759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:48:51.239566 9052 heartbeater.cc:503] Master 127.13.158.126:35877 was elected leader, sending a full tablet report...
I20260808 11:48:51.247509 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.224538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.13.158.67:43577)
0808 11:48:51.236207 (+ 11669us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.236217 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.237008 (+ 791us) server_negotiation.cc:407] Connection header received
0808 11:48:51.237126 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.237132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.237385 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.237584 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:51.240228 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.241779 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.244229 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.245672 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.245848 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.246218 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.246909 (+ 691us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.247109 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11461,"thread_start_us":11288,"threads_started":1}
I20260808 11:48:51.247547 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.237711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.13.158.66:37403)
0808 11:48:51.239962 (+ 2251us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.239982 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.240010 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:48:51.240113 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.240121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.240342 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.240544 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:51.241502 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.243048 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.245309 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.246602 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.246796 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.247035 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.247188 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.247311 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2071,"thread_start_us":3801,"threads_started":1}
I20260808 11:48:51.247716 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.237447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:37403 (local address 127.0.0.1:51406)
0808 11:48:51.238615 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.238634 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:51.238785 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:51.240801 (+ 2016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:51.240805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:51.240826 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:51.241291 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.241304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.243256 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.243262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.245115 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.245126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.246826 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.246835 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.246999 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.247030 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:48:51.247121 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 11:48:51.247336 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"mutex_wait_us":3,"thread_start_us":884,"threads_started":1}
I20260808 11:48:51.248358 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.224216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:43577 (local address 127.0.0.1:43616)
0808 11:48:51.236367 (+ 12151us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.236468 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:51.236761 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:51.238987 (+ 2226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:51.238994 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:51.239019 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:51.239993 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.240007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.241985 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.241990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.244010 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.244023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.246362 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.246368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.246588 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.246651 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 11:48:51.247831 (+ 1180us) client_negotiation.cc:240] Negotiation successful
0808 11:48:51.247972 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11714,"mutex_wait_us":3,"spinlock_wait_cycles":896,"thread_start_us":12028,"threads_started":1}
I20260808 11:48:51.248713 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.224353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:42840)
0808 11:48:51.236562 (+ 12209us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.236582 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:51.236760 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:51.239097 (+ 2337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:51.239105 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:51.239162 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:51.239656 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.239673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.241843 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.241849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.244472 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.244487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.246678 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.246684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.248037 (+ 1353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.248070 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:48:51.248164 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 11:48:51.248321 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12072,"mutex_wait_us":1198,"spinlock_wait_cycles":19712,"thread_start_us":12580,"threads_started":1}
I20260808 11:48:51.248762 9468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.224621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.13.158.65:37397)
0808 11:48:51.238354 (+ 13733us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.238366 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.238401 (+ 35us) server_negotiation.cc:407] Connection header received
0808 11:48:51.238516 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.238526 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.238812 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.239286 (+ 474us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:51.239907 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.241678 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.244697 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.246117 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.246275 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.246543 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.248257 (+ 1714us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.248428 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13479,"thread_start_us":13335,"threads_started":1}
I20260808 11:48:51.252219 8943 tablet_service.cc:1467] Processing CreateTablet for tablet f19d7c5841d8406bae71ad3840dcb024 (DEFAULT_TABLE table=test-table [id=04d12a04d8394503ab40ff1dde7b8231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:51.252239 9167 tablet_service.cc:1467] Processing CreateTablet for tablet f19d7c5841d8406bae71ad3840dcb024 (DEFAULT_TABLE table=test-table [id=04d12a04d8394503ab40ff1dde7b8231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:51.252257 9361 tablet_service.cc:1467] Processing CreateTablet for tablet f19d7c5841d8406bae71ad3840dcb024 (DEFAULT_TABLE table=test-table [id=04d12a04d8394503ab40ff1dde7b8231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:51.253679 8943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19d7c5841d8406bae71ad3840dcb024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:51.253753 9167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19d7c5841d8406bae71ad3840dcb024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:51.253750 9361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19d7c5841d8406bae71ad3840dcb024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:51.266914 9472 tablet_bootstrap.cc:492] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Bootstrap starting.
I20260808 11:48:51.268953 9474 tablet_bootstrap.cc:492] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Bootstrap starting.
I20260808 11:48:51.270913 9472 tablet_bootstrap.cc:654] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:51.271468 9475 tablet_bootstrap.cc:492] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Bootstrap starting.
I20260808 11:48:51.272498 9474 tablet_bootstrap.cc:654] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:51.272543 9472 log.cc:824] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:51.273790 9474 log.cc:824] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:51.275023 9475 tablet_bootstrap.cc:654] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:51.276222 9472 tablet_bootstrap.cc:492] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: No bootstrap required, opened a new log
I20260808 11:48:51.276208 9475 log.cc:824] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:51.276511 9472 ts_tablet_manager.cc:1397] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260808 11:48:51.277715 9474 tablet_bootstrap.cc:492] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: No bootstrap required, opened a new log
I20260808 11:48:51.278030 9474 ts_tablet_manager.cc:1397] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260808 11:48:51.279799 9475 tablet_bootstrap.cc:492] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: No bootstrap required, opened a new log
I20260808 11:48:51.280071 9475 ts_tablet_manager.cc:1397] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 11:48:51.287405 9475 raft_consensus.cc:348] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.287808 9475 raft_consensus.cc:374] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:51.287614 9472 raft_consensus.cc:348] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.287954 9475 raft_consensus.cc:729] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 022d24dc08aa4349b9c7f64fd6989bf8, State: Initialized, Role: FOLLOWER
I20260808 11:48:51.288033 9472 raft_consensus.cc:374] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:51.288136 9472 raft_consensus.cc:729] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 812fee04617646d3af2db01252b5149c, State: Initialized, Role: FOLLOWER
I20260808 11:48:51.288635 9475 consensus_queue.cc:260] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.288651 9474 raft_consensus.cc:348] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.288817 9472 consensus_queue.cc:260] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.289077 9474 raft_consensus.cc:374] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:51.289217 9474 raft_consensus.cc:729] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b518a23f3754db7bc0be970076c6e4e, State: Initialized, Role: FOLLOWER
I20260808 11:48:51.289986 9474 consensus_queue.cc:260] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.290812 9472 ts_tablet_manager.cc:1428] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260808 11:48:51.291118 9260 heartbeater.cc:503] Master 127.13.158.126:35877 was elected leader, sending a full tablet report...
I20260808 11:48:51.291746 9474 ts_tablet_manager.cc:1428] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 11:48:51.294669 9450 heartbeater.cc:503] Master 127.13.158.126:35877 was elected leader, sending a full tablet report...
I20260808 11:48:51.296139 9475 ts_tablet_manager.cc:1428] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260808 11:48:51.421746 9451 tablet.cc:2369] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:48:51.440949 9261 tablet.cc:2369] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:51.455547 9479 raft_consensus.cc:482] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:51.455907 9479 raft_consensus.cc:504] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.457693 9479 leader_election.cc:288] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 812fee04617646d3af2db01252b5149c (127.13.158.65:37397), 3b518a23f3754db7bc0be970076c6e4e (127.13.158.66:37403)
I20260808 11:48:51.463608 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.458121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.13.158.67:56055)
0808 11:48:51.458413 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.458452 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:51.458664 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:51.459302 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:51.459306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:51.459334 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:51.459486 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.459494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.460517 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.460523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.461793 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.461804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.462910 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.462917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.463080 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.463103 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:48:51.463222 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 11:48:51.463356 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 11:48:51.463608 9468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.458164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:56055 (local address 127.13.158.65:37397)
0808 11:48:51.458416 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.458423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.458767 (+ 344us) server_negotiation.cc:407] Connection header received
0808 11:48:51.458853 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.458865 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.458996 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.459098 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:51.459660 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.460362 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.461966 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.462700 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.462932 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.463001 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.463242 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.463364 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 11:48:51.464625 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.458903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:58771 (local address 127.13.158.66:37403)
0808 11:48:51.459150 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.459156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.459529 (+ 373us) server_negotiation.cc:407] Connection header received
0808 11:48:51.459744 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.459748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.459817 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.459923 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:51.460488 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.461161 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.462844 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.463546 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.463788 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.463848 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.464339 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.464442 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 11:48:51.464679 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.458585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:37403 (local address 127.13.158.67:58771)
0808 11:48:51.459411 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.459423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:51.459573 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:51.459939 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:51.459962 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:51.459984 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:51.460315 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.460323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.461320 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.461328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.462597 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:51.462608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.463908 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.463917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:51.464069 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.464109 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 11:48:51.464283 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 11:48:51.464409 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":489,"threads_started":1}
I20260808 11:48:51.464696 8972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19d7c5841d8406bae71ad3840dcb024" candidate_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "812fee04617646d3af2db01252b5149c" is_pre_election: true
I20260808 11:48:51.465377 8972 raft_consensus.cc:2454] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 022d24dc08aa4349b9c7f64fd6989bf8 in term 0.
I20260808 11:48:51.465749 9197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19d7c5841d8406bae71ad3840dcb024" candidate_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b518a23f3754db7bc0be970076c6e4e" is_pre_election: true
I20260808 11:48:51.466171 9292 leader_election.cc:302] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022d24dc08aa4349b9c7f64fd6989bf8, 812fee04617646d3af2db01252b5149c; no voters:
I20260808 11:48:51.466316 9197 raft_consensus.cc:2454] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 022d24dc08aa4349b9c7f64fd6989bf8 in term 0.
W20260808 11:48:51.466382 9055 tablet.cc:2369] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:51.466622 9479 raft_consensus.cc:2781] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:48:51.466812 9479 raft_consensus.cc:482] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:48:51.466933 9479 raft_consensus.cc:3037] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:51.469110 9479 raft_consensus.cc:504] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.469753 9479 leader_election.cc:288] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [CANDIDATE]: Term 1 election: Requested vote from peers 812fee04617646d3af2db01252b5149c (127.13.158.65:37397), 3b518a23f3754db7bc0be970076c6e4e (127.13.158.66:37403)
I20260808 11:48:51.470212 8972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19d7c5841d8406bae71ad3840dcb024" candidate_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "812fee04617646d3af2db01252b5149c"
I20260808 11:48:51.470256 9197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19d7c5841d8406bae71ad3840dcb024" candidate_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b518a23f3754db7bc0be970076c6e4e"
I20260808 11:48:51.470429 8972 raft_consensus.cc:3037] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:51.470431 9197 raft_consensus.cc:3037] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:51.473701 8972 raft_consensus.cc:2454] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 022d24dc08aa4349b9c7f64fd6989bf8 in term 1.
I20260808 11:48:51.473719 9197 raft_consensus.cc:2454] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 022d24dc08aa4349b9c7f64fd6989bf8 in term 1.
I20260808 11:48:51.474092 9292 leader_election.cc:302] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022d24dc08aa4349b9c7f64fd6989bf8, 3b518a23f3754db7bc0be970076c6e4e; no voters:
I20260808 11:48:51.474355 9479 raft_consensus.cc:2781] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:51.474865 9479 raft_consensus.cc:686] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 1 LEADER]: Becoming Leader. State: Replica: 022d24dc08aa4349b9c7f64fd6989bf8, State: Running, Role: LEADER
I20260808 11:48:51.475433 9479 consensus_queue.cc:237] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } }
I20260808 11:48:51.480423 8757 catalog_manager.cc:5495] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 022d24dc08aa4349b9c7f64fd6989bf8 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022d24dc08aa4349b9c7f64fd6989bf8" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 43577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 } health_report { overall_health: UNKNOWN } } }
I20260808 11:48:51.549582 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.543979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56382 (local address 127.13.158.67:43577)
0808 11:48:51.544214 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.544220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.544239 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:48:51.544416 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.544419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.544507 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.544687 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:48:51.545148 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.545881 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.547332 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.547741 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.547883 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.548472 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:48:51.548953 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:48:51.549146 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.549310 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.549350 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:48:51.558069 8972 raft_consensus.cc:1261] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 1 FOLLOWER]: Refusing update from remote peer 022d24dc08aa4349b9c7f64fd6989bf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:51.558069 9197 raft_consensus.cc:1261] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 1 FOLLOWER]: Refusing update from remote peer 022d24dc08aa4349b9c7f64fd6989bf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:51.559100 9483 consensus_queue.cc:1035] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b518a23f3754db7bc0be970076c6e4e" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 37403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:51.559507 9479 consensus_queue.cc:1035] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "812fee04617646d3af2db01252b5149c" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 37397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:51.574012 9487 mvcc.cc:205] Tried to move back new op lower bound from 7316233140451270656 to 7316233140128038912. Current Snapshot: MvccSnapshot[applied={T|T < 7316233140451270656}]
I20260808 11:48:51.585790 8759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:51.586086 8759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:51.589668 8759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3476238d275486aa76bcb204d45b874: Sending DeleteTablet for 3 replicas of tablet f19d7c5841d8406bae71ad3840dcb024
I20260808 11:48:51.590807 9361 tablet_service.cc:1514] Processing DeleteTablet for tablet f19d7c5841d8406bae71ad3840dcb024 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:51 UTC) from {username='slave'} at 127.0.0.1:43616
I20260808 11:48:51.591094 8943 tablet_service.cc:1514] Processing DeleteTablet for tablet f19d7c5841d8406bae71ad3840dcb024 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:51 UTC) from {username='slave'} at 127.0.0.1:42840
I20260808 11:48:51.591216 9167 tablet_service.cc:1514] Processing DeleteTablet for tablet f19d7c5841d8406bae71ad3840dcb024 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:51 UTC) from {username='slave'} at 127.0.0.1:51406
I20260808 11:48:51.592737 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 8845
I20260808 11:48:51.599690 9499 tablet_replica.cc:330] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: stopping tablet replica
I20260808 11:48:51.600175 9499 raft_consensus.cc:2229] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:51.600728 9499 raft_consensus.cc:2258] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:51.603452 9499 ts_tablet_manager.cc:1905] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:51.604631 9498 tablet_replica.cc:330] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: stopping tablet replica
I20260808 11:48:51.605203 9503 tablet_replica.cc:330] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: stopping tablet replica
I20260808 11:48:51.605762 9498 raft_consensus.cc:2229] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:48:51.606297 9503 raft_consensus.cc:2229] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:51.607395 9498 raft_consensus.cc:2258] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:51.607944 9503 raft_consensus.cc:2258] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:51.608979 9468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.595050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.13.158.65:37397)
0808 11:48:51.595247 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.595253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.600250 (+ 4997us) server_negotiation.cc:407] Connection header received
0808 11:48:51.600467 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.600472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.600567 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.600676 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:51.601408 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.602521 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.604806 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.605510 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.605621 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.607274 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:51.607287 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:51.607303 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:51.608023 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:51.608097 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:51.608134 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:51.608220 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.608467 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.608593 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:48:51.610137 9044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:51.611974 9498 ts_tablet_manager.cc:1905] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:51.615963 9503 ts_tablet_manager.cc:1905] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:51.622957 9503 ts_tablet_manager.cc:1918] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:51.623122 9503 log.cc:1196] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/f19d7c5841d8406bae71ad3840dcb024
I20260808 11:48:51.623726 9503 ts_tablet_manager.cc:1939] T f19d7c5841d8406bae71ad3840dcb024 P 3b518a23f3754db7bc0be970076c6e4e: Deleting consensus metadata
I20260808 11:48:51.625831 8735 catalog_manager.cc:4841] TS 3b518a23f3754db7bc0be970076c6e4e (127.13.158.66:37403): tablet f19d7c5841d8406bae71ad3840dcb024 (table test-table [id=04d12a04d8394503ab40ff1dde7b8231]) successfully deleted
I20260808 11:48:51.626045 9498 ts_tablet_manager.cc:1918] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:51.626219 9498 log.cc:1196] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/f19d7c5841d8406bae71ad3840dcb024
I20260808 11:48:51.626961 9498 ts_tablet_manager.cc:1939] T f19d7c5841d8406bae71ad3840dcb024 P 022d24dc08aa4349b9c7f64fd6989bf8: Deleting consensus metadata
I20260808 11:48:51.628791 8734 catalog_manager.cc:4841] TS 022d24dc08aa4349b9c7f64fd6989bf8 (127.13.158.67:43577): tablet f19d7c5841d8406bae71ad3840dcb024 (table test-table [id=04d12a04d8394503ab40ff1dde7b8231]) successfully deleted
I20260808 11:48:51.715688 9499 ts_tablet_manager.cc:1918] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:51.715976 9499 log.cc:1196] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/f19d7c5841d8406bae71ad3840dcb024
I20260808 11:48:51.716440 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 8845
I20260808 11:48:51.716629 9499 ts_tablet_manager.cc:1939] T f19d7c5841d8406bae71ad3840dcb024 P 812fee04617646d3af2db01252b5149c: Deleting consensus metadata
I20260808 11:48:51.742244 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 9059
W20260808 11:48:51.742275 8735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.158.65:37397 (error 108)
I20260808 11:48:51.744279 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.743770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:39978)
0808 11:48:51.743971 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.744104 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 11:48:51.744524 8735 catalog_manager.cc:4584] TS 812fee04617646d3af2db01252b5149c (127.13.158.65:37397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f19d7c5841d8406bae71ad3840dcb024: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
I20260808 11:48:51.749681 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.743417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.13.158.66:37403)
0808 11:48:51.743630 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.743639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.743660 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:48:51.743727 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.743732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.743850 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.743954 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:51.744385 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.745288 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.746542 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.747010 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.747177 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.748292 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:51.748301 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:51.748315 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:51.748880 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:51.748925 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:51.748944 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:51.749083 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.749231 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.749340 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:48:51.750310 9254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:51.798051 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.797604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:39980)
0808 11:48:51.797812 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.797910 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 11:48:51.799293 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.798936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:39996)
0808 11:48:51.799092 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.799183 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 11:48:51.860028 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.859525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:40012)
0808 11:48:51.859761 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.859849 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 11:48:51.861116 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.860773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:40014)
0808 11:48:51.860932 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.861010 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 11:48:51.892108 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 9059
I20260808 11:48:51.923489 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 9264
I20260808 11:48:51.931804 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.924399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56396 (local address 127.13.158.67:43577)
0808 11:48:51.924653 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:51.924659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:51.924683 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:48:51.924789 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:51.924795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:51.924916 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:51.925037 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:51.925497 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.926794 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.928221 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:51.928906 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:51.929072 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:51.930120 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:51.930132 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:51.930151 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:51.930987 (+ 836us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:51.931047 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:51.931073 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:51.931184 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:51.931325 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 11:48:51.931422 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 11:48:51.932668 9439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:51.941383 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.940861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:40024)
0808 11:48:51.941076 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.941198 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 11:48:51.942552 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:51.942028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:40040)
0808 11:48:51.942251 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:51.942381 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 11:48:52.059553 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 9264
I20260808 11:48:52.087723 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 8708
I20260808 11:48:52.088912 8801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:52.198280 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:52.197640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:37397 (local address 127.0.0.1:40056)
0808 11:48:52.197815 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:52.198037 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.65:37397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 11:48:52.198370 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 8708
2026-08-08T11:48:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3068 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:48:52 dist-test-slave-39l8 krb5kdc[9518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:48:52 dist-test-slave-39l8 krb5kdc[9518](info): set up 2 sockets
Aug 08 11:48:52 dist-test-slave-39l8 krb5kdc[9518](info): commencing operation
krb5kdc: starting...
W20260808 11:48:53.522089 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:48:53 dist-test-slave-39l8 krb5kdc[9518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:48:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:48:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:48:53.828114 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44439
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:44439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:54.168727 9538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:54.169034 9538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:54.169126 9538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:54.176355 9538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:48:54.176460 9538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:54.176509 9538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:54.176568 9538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:48:54.176609 9538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:48:54.186017 9538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:44439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:54.187330 9538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:54.188589 9538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:54.200290 9548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:54.200348 9549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:54.201076 9554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:54.203573 9538 server_base.cc:1034] running on GCE node
I20260808 11:48:54.204721 9538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:54.206874 9538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:54.208127 9538 hybrid_clock.cc:648] HybridClock initialized: now 1786189734208069 us; error 32 us; skew 500 ppm
I20260808 11:48:54.208640 9538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:54 dist-test-slave-39l8 krb5kdc[9518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189734, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:54.212236 9538 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:48:54.213729 9538 webserver.cc:460] Webserver started at http://127.13.158.126:39091/ using document root <none> and password file <none>
I20260808 11:48:54.214488 9538 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:54.214633 9538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:54.214970 9538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:54.218005 9538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "064fc87452304e84a4d74ad520fab002"
format_stamp: "Formatted at 2026-08-08 11:48:54 on dist-test-slave-39l8"
server_key: "66c4123db36a7a8d95f058e143537dee"
server_key_iv: "5be70a64680517c200867e1e6c718322"
server_key_version: "encryptionkey@0"
I20260808 11:48:54.218830 9538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "064fc87452304e84a4d74ad520fab002"
format_stamp: "Formatted at 2026-08-08 11:48:54 on dist-test-slave-39l8"
server_key: "66c4123db36a7a8d95f058e143537dee"
server_key_iv: "5be70a64680517c200867e1e6c718322"
server_key_version: "encryptionkey@0"
I20260808 11:48:54.224017 9538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:48:54.227579 9561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:54.228964 9538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:48:54.229166 9538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "064fc87452304e84a4d74ad520fab002"
format_stamp: "Formatted at 2026-08-08 11:48:54 on dist-test-slave-39l8"
server_key: "66c4123db36a7a8d95f058e143537dee"
server_key_iv: "5be70a64680517c200867e1e6c718322"
server_key_version: "encryptionkey@0"
I20260808 11:48:54.229393 9538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:54.249621 9538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:54.253602 9538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:54.253885 9538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:54.272539 9538 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:44439
I20260808 11:48:54.272562 9639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:44439 every 8 connection(s)
I20260808 11:48:54.273923 9538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:48:54.277546 9640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:54.278211 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 9538
I20260808 11:48:54.278394 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:48:54.279024 13945 external_mini_cluster.cc:1428] Setting key 4cee3817994050a7bfda72cb697957c4
I20260808 11:48:54.291433 9640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002: Bootstrap starting.
Aug 08 11:48:54 dist-test-slave-39l8 krb5kdc[9518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:54.295220 9640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:54.296275 9640 log.cc:824] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:54.299476 9640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002: No bootstrap required, opened a new log
I20260808 11:48:54.303190 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:54.280946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.13.158.126:44439)
0808 11:48:54.281761 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:54.281772 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:54.281814 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:48:54.283847 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:54.283870 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:54.284257 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:54.284565 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:54.286385 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:54.287711 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:54.289542 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:54.290148 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:54.290268 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:54.295072 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:54.295079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:54.295092 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:54.295128 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:54.298832 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:54.299431 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:54.299440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:54.299450 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:54.299657 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:54.300003 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:54.300008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:54.300011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:54.300686 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:54.300967 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:54.301342 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 11:48:54.301789 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":390,"threads_started":1}
I20260808 11:48:54.308049 9640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064fc87452304e84a4d74ad520fab002" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44439 } }
I20260808 11:48:54.308363 9640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:54.308436 9640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064fc87452304e84a4d74ad520fab002, State: Initialized, Role: FOLLOWER
I20260808 11:48:54.308897 9640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064fc87452304e84a4d74ad520fab002" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44439 } }
I20260808 11:48:54.309082 9640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:48:54.309175 9640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:48:54.309298 9640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:54.311388 9640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064fc87452304e84a4d74ad520fab002" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44439 } }
I20260808 11:48:54.311820 9640 leader_election.cc:302] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 064fc87452304e84a4d74ad520fab002; no voters:
I20260808 11:48:54.312383 9640 leader_election.cc:288] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:48:54.312513 9646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:54.313263 9646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [term 1 LEADER]: Becoming Leader. State: Replica: 064fc87452304e84a4d74ad520fab002, State: Running, Role: LEADER
I20260808 11:48:54.313805 9646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064fc87452304e84a4d74ad520fab002" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44439 } }
I20260808 11:48:54.314574 9640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:48:54.317109 9648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 064fc87452304e84a4d74ad520fab002. Latest consensus state: current_term: 1 leader_uuid: "064fc87452304e84a4d74ad520fab002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064fc87452304e84a4d74ad520fab002" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44439 } } }
I20260808 11:48:54.317593 9648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:54.317119 9647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "064fc87452304e84a4d74ad520fab002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064fc87452304e84a4d74ad520fab002" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44439 } } }
I20260808 11:48:54.318331 9647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002 [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:54.319856 9657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:48:54.326081 9657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:48:54.334609 9657 catalog_manager.cc:1347] Generated new cluster ID: 07c8575104f14e20a2e91a8c8b1af115
I20260808 11:48:54.334750 9657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:48:54.355278 9657 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:48:54.357163 9657 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:48:54.373055 9657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002: Generated new TSK 0
I20260808 11:48:54.374150 9657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
Entry for principal kudu/127.13.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab.
I20260808 11:48:54.451279 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44439
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:54.798106 9680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:54.798381 9680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:54.798478 9680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:54.805760 9680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:54.805868 9680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:54.805995 9680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:48:54.815954 9680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.65.keytab
--principal=kudu/127.13.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:54.817164 9680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:54.818456 9680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:54.831395 9692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:54.831470 9685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:54.831429 9688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:54.832304 9680 server_base.cc:1034] running on GCE node
I20260808 11:48:54.832908 9680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:54.833840 9680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:54.835079 9680 hybrid_clock.cc:648] HybridClock initialized: now 1786189734834997 us; error 59 us; skew 500 ppm
I20260808 11:48:54.835399 9680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:54 dist-test-slave-39l8 krb5kdc[9518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189734, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:54.838439 9680 init.cc:377] Logged in from keytab as kudu/127.13.158.65@KRBTEST.COM (short username kudu)
I20260808 11:48:54.839934 9680 webserver.cc:460] Webserver started at http://127.13.158.65:45215/ using document root <none> and password file <none>
I20260808 11:48:54.840649 9680 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:54.840796 9680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:54.841107 9680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:54.844074 9680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "8a42b499269c45c29870774d96fb9405"
format_stamp: "Formatted at 2026-08-08 11:48:54 on dist-test-slave-39l8"
server_key: "5dff705771abe1cac380024f769378af"
server_key_iv: "81704b32b300ddf1ae195e722f271a55"
server_key_version: "encryptionkey@0"
I20260808 11:48:54.844841 9680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "8a42b499269c45c29870774d96fb9405"
format_stamp: "Formatted at 2026-08-08 11:48:54 on dist-test-slave-39l8"
server_key: "5dff705771abe1cac380024f769378af"
server_key_iv: "81704b32b300ddf1ae195e722f271a55"
server_key_version: "encryptionkey@0"
I20260808 11:48:54.849907 9680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 11:48:54.853379 9699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:54.854609 9680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:48:54.854826 9680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "8a42b499269c45c29870774d96fb9405"
format_stamp: "Formatted at 2026-08-08 11:48:54 on dist-test-slave-39l8"
server_key: "5dff705771abe1cac380024f769378af"
server_key_iv: "81704b32b300ddf1ae195e722f271a55"
server_key_version: "encryptionkey@0"
I20260808 11:48:54.855026 9680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:54.903524 9680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:54.907533 9680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:54.907821 9680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:54.908807 9680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:54.910231 9680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:54.910336 9680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:54.910511 9680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:54.910596 9680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:54.944510 9680 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:38395
I20260808 11:48:54.944558 9859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:38395 every 8 connection(s)
I20260808 11:48:54.945855 9680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:48:54.952852 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 9680
I20260808 11:48:54.953047 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:48:54.953663 13945 external_mini_cluster.cc:1428] Setting key 77d55a7d5b81cbe0e9aa28655cb95285
Aug 08 11:48:54 dist-test-slave-39l8 krb5kdc[9518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189734, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.65@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:54.967572 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:54.948339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:48373 (local address 127.13.158.126:44439)
0808 11:48:54.948818 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:54.948823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:54.949682 (+ 859us) server_negotiation.cc:407] Connection header received
0808 11:48:54.951526 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:54.951531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:54.951614 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:54.951737 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:54.953424 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:54.954497 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:54.956247 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:54.956964 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:54.957126 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:54.960726 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:54.960734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:54.960741 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:54.960786 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:54.964233 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:54.964931 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:54.964939 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:54.964945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:54.965113 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:54.965520 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:54.965528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:54.965534 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:54.965925 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:54.966109 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:54.966786 (+ 677us) server_negotiation.cc:299] Negotiation successful
0808 11:48:54.967083 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
I20260808 11:48:54.968228 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:54.948819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44439 (local address 127.13.158.65:48373)
0808 11:48:54.949522 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:54.949565 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:54.951271 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:54.951988 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:54.951995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:54.952501 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:54.953178 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:54.953203 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:54.954641 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:54.954645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:54.956105 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:54.956117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:54.957171 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:54.957175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:54.957373 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:54.958137 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:54.958166 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:54.960494 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:54.964363 (+ 3869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:54.964370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:54.964381 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:54.964799 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:54.965217 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:54.965220 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:54.965222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:54.965389 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:54.966051 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:54.966057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:54.966363 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 11:48:54.966586 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 11:48:54.967041 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":291,"threads_started":1}
I20260808 11:48:54.969836 9861 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44439
I20260808 11:48:54.970186 9861 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:54.970994 9861 heartbeater.cc:511] Master 127.13.158.126:44439 requested a full tablet report, sending...
I20260808 11:48:54.973132 9587 ts_manager.cc:194] Registered new tserver with Master: 8a42b499269c45c29870774d96fb9405 (127.13.158.65:38395)
I20260808 11:48:54.974556 9587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.65@KRBTEST.COM'} at 127.13.158.65:48373
WARNING: no policy specified for kudu/127.13.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
Entry for principal kudu/127.13.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab.
I20260808 11:48:55.021935 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44439
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:55.357996 9872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:55.358314 9872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:55.358474 9872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:55.365796 9872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:55.365952 9872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:55.366122 9872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:48:55.376111 9872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.66.keytab
--principal=kudu/127.13.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:55.377415 9872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:55.378784 9872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:55.392017 9879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:55.392531 9882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:55.392599 9877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:55.392915 9872 server_base.cc:1034] running on GCE node
I20260808 11:48:55.393509 9872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:55.394387 9872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:55.395582 9872 hybrid_clock.cc:648] HybridClock initialized: now 1786189735395527 us; error 50 us; skew 500 ppm
I20260808 11:48:55.395854 9872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:55 dist-test-slave-39l8 krb5kdc[9518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189735, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:55.398968 9872 init.cc:377] Logged in from keytab as kudu/127.13.158.66@KRBTEST.COM (short username kudu)
I20260808 11:48:55.400312 9872 webserver.cc:460] Webserver started at http://127.13.158.66:45431/ using document root <none> and password file <none>
I20260808 11:48:55.400954 9872 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:55.401060 9872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:55.401331 9872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:55.404129 9872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "b3247e2a6be4468fb33118a232860fff"
format_stamp: "Formatted at 2026-08-08 11:48:55 on dist-test-slave-39l8"
server_key: "d3a577af7105f9f8824784ce7b163765"
server_key_iv: "fc297e17fec190e1f0d3a402909f51c5"
server_key_version: "encryptionkey@0"
I20260808 11:48:55.404844 9872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "b3247e2a6be4468fb33118a232860fff"
format_stamp: "Formatted at 2026-08-08 11:48:55 on dist-test-slave-39l8"
server_key: "d3a577af7105f9f8824784ce7b163765"
server_key_iv: "fc297e17fec190e1f0d3a402909f51c5"
server_key_version: "encryptionkey@0"
I20260808 11:48:55.410090 9872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:48:55.413522 9889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:55.414875 9872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:48:55.415055 9872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "b3247e2a6be4468fb33118a232860fff"
format_stamp: "Formatted at 2026-08-08 11:48:55 on dist-test-slave-39l8"
server_key: "d3a577af7105f9f8824784ce7b163765"
server_key_iv: "fc297e17fec190e1f0d3a402909f51c5"
server_key_version: "encryptionkey@0"
I20260808 11:48:55.415268 9872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:55.453233 9872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:55.457691 9872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:55.457998 9872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:55.459031 9872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:55.460387 9872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:55.460503 9872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:55.460654 9872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:55.460727 9872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:55.494551 9872 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:39251
I20260808 11:48:55.494621 10053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:39251 every 8 connection(s)
I20260808 11:48:55.495921 9872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:48:55.503774 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 9872
I20260808 11:48:55.503949 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:48:55.504395 13945 external_mini_cluster.cc:1428] Setting key f98f5d855b2fd3d2a86daee4513c1d4f
Aug 08 11:48:55 dist-test-slave-39l8 krb5kdc[9518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189735, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.66@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:55.517872 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:55.498502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:50375 (local address 127.13.158.126:44439)
0808 11:48:55.499013 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:55.499018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:55.499985 (+ 967us) server_negotiation.cc:407] Connection header received
0808 11:48:55.501826 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:55.501832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:55.501939 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:55.502054 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:55.503589 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:55.504443 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:55.506496 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:55.507006 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:55.507140 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:55.510661 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:55.510670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:55.510677 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:55.510743 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:55.514376 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:55.515216 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:55.515224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:55.515230 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:55.515401 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:55.515848 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:55.515856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:55.515862 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:55.516281 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:55.516457 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:55.517121 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 11:48:55.517381 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":281,"threads_started":1}
I20260808 11:48:55.518491 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:55.499100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44439 (local address 127.13.158.66:50375)
0808 11:48:55.499809 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:55.499858 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:55.501560 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:55.502288 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:55.502296 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:55.502882 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:55.503424 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:55.503440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:55.504598 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:55.504602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:55.506355 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:55.506370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:55.507236 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:55.507244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:55.507470 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:55.508217 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:55.508240 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:55.510432 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:55.514523 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:55.514530 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:55.514542 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:55.515067 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:55.515517 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:55.515520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:55.515522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:55.515711 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:55.516405 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:55.516411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:55.516726 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 11:48:55.516952 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 11:48:55.517345 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":279,"threads_started":1}
I20260808 11:48:55.520064 10054 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44439
I20260808 11:48:55.520407 10054 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:55.521317 10054 heartbeater.cc:511] Master 127.13.158.126:44439 requested a full tablet report, sending...
I20260808 11:48:55.522751 9587 ts_manager.cc:194] Registered new tserver with Master: b3247e2a6be4468fb33118a232860fff (127.13.158.66:39251)
I20260808 11:48:55.523550 9587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.66@KRBTEST.COM'} at 127.13.158.66:50375
WARNING: no policy specified for kudu/127.13.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
Entry for principal kudu/127.13.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab.
I20260808 11:48:55.577514 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44439
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:48:55.921378 10061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:55.921705 10061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:55.921892 10061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:55.929100 10061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:55.929241 10061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:55.929419 10061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:48:55.939139 10061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.67.keytab
--principal=kudu/127.13.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:55.940375 10061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:55.941674 10061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:55.956573 10070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:55.956547 10074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:55.956502 10071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:55.957353 10061 server_base.cc:1034] running on GCE node
I20260808 11:48:55.957991 10061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:55.958969 10061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:55.960196 10061 hybrid_clock.cc:648] HybridClock initialized: now 1786189735960162 us; error 36 us; skew 500 ppm
I20260808 11:48:55.960486 10061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:48:55 dist-test-slave-39l8 krb5kdc[9518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189735, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:48:55.963657 10061 init.cc:377] Logged in from keytab as kudu/127.13.158.67@KRBTEST.COM (short username kudu)
I20260808 11:48:55.964998 10061 webserver.cc:460] Webserver started at http://127.13.158.67:41119/ using document root <none> and password file <none>
I20260808 11:48:55.965689 10061 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:55.965831 10061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:55.966148 10061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:55.969043 10061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "970a191ac026492ba041bee75db7ba40"
format_stamp: "Formatted at 2026-08-08 11:48:55 on dist-test-slave-39l8"
server_key: "da3ac139e50b301c41a37e6afc563786"
server_key_iv: "d333dea0708d07b3219cdb49652e14e1"
server_key_version: "encryptionkey@0"
I20260808 11:48:55.969791 10061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "970a191ac026492ba041bee75db7ba40"
format_stamp: "Formatted at 2026-08-08 11:48:55 on dist-test-slave-39l8"
server_key: "da3ac139e50b301c41a37e6afc563786"
server_key_iv: "d333dea0708d07b3219cdb49652e14e1"
server_key_version: "encryptionkey@0"
I20260808 11:48:55.974879 10061 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 11:48:55.977381 9861 heartbeater.cc:503] Master 127.13.158.126:44439 was elected leader, sending a full tablet report...
I20260808 11:48:55.978298 10081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:55.979764 10061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:48:55.979926 10061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "970a191ac026492ba041bee75db7ba40"
format_stamp: "Formatted at 2026-08-08 11:48:55 on dist-test-slave-39l8"
server_key: "da3ac139e50b301c41a37e6afc563786"
server_key_iv: "d333dea0708d07b3219cdb49652e14e1"
server_key_version: "encryptionkey@0"
I20260808 11:48:55.980114 10061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:55.995311 10061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:55.999627 10061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:55.999893 10061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:56.000828 10061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:56.002141 10061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:56.002215 10061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:56.002393 10061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:56.002444 10061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:56.039322 10061 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:38369
I20260808 11:48:56.039373 10253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:38369 every 8 connection(s)
I20260808 11:48:56.040647 10061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:48:56.050822 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 10061
I20260808 11:48:56.051009 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:48:56.051514 13945 external_mini_cluster.cc:1428] Setting key f010eb13cf211a366b895440d67c1dac
Aug 08 11:48:56 dist-test-slave-39l8 krb5kdc[9518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189735, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.67@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:48:56.061259 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.043214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:38609 (local address 127.13.158.126:44439)
0808 11:48:56.043688 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.043692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.044521 (+ 829us) server_negotiation.cc:407] Connection header received
0808 11:48:56.046335 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.046341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.046432 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.046549 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:56.048071 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.048870 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.050519 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.051054 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.051245 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.054987 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:56.054991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:56.054995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:56.055034 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:56.058269 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.059079 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.059085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.059088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.059255 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.059745 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.059748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.059750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.060012 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:56.060188 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.060811 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.060969 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":250,"threads_started":1}
I20260808 11:48:56.062170 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.043630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44439 (local address 127.13.158.67:38609)
0808 11:48:56.044322 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:56.044387 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:56.046070 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:56.046822 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:56.046830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:56.047339 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:56.047879 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.047896 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.049049 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.049060 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.050359 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.050372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.051306 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.051317 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.051580 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.052299 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:48:56.052322 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:56.054792 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:56.058411 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:56.058421 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:56.058438 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:56.058949 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:56.059392 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:48:56.059396 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:48:56.059398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:48:56.059640 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:48:56.060166 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:56.060172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:56.060510 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 11:48:56.060730 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 11:48:56.061094 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":264,"threads_started":1}
I20260808 11:48:56.063738 10255 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44439
I20260808 11:48:56.064025 10255 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:56.064781 10255 heartbeater.cc:511] Master 127.13.158.126:44439 requested a full tablet report, sending...
I20260808 11:48:56.066392 9587 ts_manager.cc:194] Registered new tserver with Master: 970a191ac026492ba041bee75db7ba40 (127.13.158.67:38369)
I20260808 11:48:56.067041 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:48:56.067412 9587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.158.67@KRBTEST.COM'} at 127.13.158.67:38609
I20260808 11:48:56.083985 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.071348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55364 (local address 127.13.158.126:44439)
0808 11:48:56.071502 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.071506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.071949 (+ 443us) server_negotiation.cc:407] Connection header received
0808 11:48:56.072169 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.072173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.072248 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.072348 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:56.073726 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.074523 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.075814 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.076244 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.076373 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.078802 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:56.078807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:56.078810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:56.078842 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:56.081284 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.081938 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.081944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.081947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.082072 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.082427 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.082435 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.082438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.082809 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:56.083044 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.083446 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.083666 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:48:56.087893 9587 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:48:56.094625 9587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:48:56.119783 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.107421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:38369 (local address 127.0.0.1:45730)
0808 11:48:56.108732 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:56.108794 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:56.108975 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:56.110714 (+ 1739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:56.110739 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:56.110767 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:56.111220 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.111242 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.113391 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.113400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.116878 (+ 3478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.116890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.118690 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.118697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.118933 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.118993 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 11:48:56.119106 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 11:48:56.119320 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1208,"mutex_wait_us":59,"thread_start_us":873,"threads_started":1}
I20260808 11:48:56.120280 10280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.107464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53812 (local address 127.13.158.65:38395)
0808 11:48:56.108353 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.108363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.108913 (+ 550us) server_negotiation.cc:407] Connection header received
0808 11:48:56.109233 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.109243 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.109526 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.109688 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:56.111826 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.113307 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.117236 (+ 3929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.118776 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.118913 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.119164 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.119743 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.119915 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":517,"threads_started":1}
I20260808 11:48:56.120411 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.108068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.13.158.67:38369)
0808 11:48:56.109674 (+ 1606us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.109682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.109713 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:48:56.110257 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.110279 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.110483 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.111227 (+ 744us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:56.111512 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.113217 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.117058 (+ 3841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.118381 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.119393 (+ 1012us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.119688 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.119751 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.119913 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1429,"thread_start_us":1243,"threads_started":1}
I20260808 11:48:56.121111 10278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.107152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:38395 (local address 127.0.0.1:53812)
0808 11:48:56.108732 (+ 1580us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:56.108794 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:56.108998 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:56.110025 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:56.110032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:56.110065 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:56.111601 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.111613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.113477 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.113484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.117051 (+ 3567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.117065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.119314 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.119324 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.119509 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.119540 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:48:56.120661 (+ 1121us) client_negotiation.cc:240] Negotiation successful
0808 11:48:56.120783 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1298,"mutex_wait_us":924,"thread_start_us":1348,"threads_started":1}
I20260808 11:48:56.124405 10157 tablet_service.cc:1467] Processing CreateTablet for tablet deca623ff9b64c9f8784888db2e6b9c6 (DEFAULT_TABLE table=test-table [id=8fee1d43336d4943950286b2eef78fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:56.124737 9769 tablet_service.cc:1467] Processing CreateTablet for tablet deca623ff9b64c9f8784888db2e6b9c6 (DEFAULT_TABLE table=test-table [id=8fee1d43336d4943950286b2eef78fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:56.125615 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.109711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:39251 (local address 127.0.0.1:54560)
0808 11:48:56.116712 (+ 7001us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:56.116737 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:56.116865 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:56.117994 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:56.117999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:56.118025 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:56.118447 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.118464 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.121417 (+ 2953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.121426 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.123062 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.123075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.124850 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.124856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.125033 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.125057 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:48:56.125162 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 11:48:56.125287 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6871,"thread_start_us":6746,"threads_started":1}
I20260808 11:48:56.125788 10283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.110004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.13.158.66:39251)
0808 11:48:56.117429 (+ 7425us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.117438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.117464 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:48:56.117557 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.117564 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.117783 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.117969 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:56.118682 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.120482 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.123410 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.124623 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.124779 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.124980 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.125270 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.125444 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7251,"thread_start_us":7108,"threads_started":1}
I20260808 11:48:56.126116 10157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deca623ff9b64c9f8784888db2e6b9c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:56.126147 9769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deca623ff9b64c9f8784888db2e6b9c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:56.128562 9969 tablet_service.cc:1467] Processing CreateTablet for tablet deca623ff9b64c9f8784888db2e6b9c6 (DEFAULT_TABLE table=test-table [id=8fee1d43336d4943950286b2eef78fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:56.129786 9969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deca623ff9b64c9f8784888db2e6b9c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:56.140574 10284 tablet_bootstrap.cc:492] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Bootstrap starting.
I20260808 11:48:56.143409 10285 tablet_bootstrap.cc:492] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Bootstrap starting.
I20260808 11:48:56.144227 10286 tablet_bootstrap.cc:492] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Bootstrap starting.
I20260808 11:48:56.144356 10284 tablet_bootstrap.cc:654] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:56.145970 10284 log.cc:824] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:56.147601 10286 tablet_bootstrap.cc:654] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:56.147696 10285 tablet_bootstrap.cc:654] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:56.148679 10285 log.cc:824] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:56.148802 10286 log.cc:824] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:56.149374 10284 tablet_bootstrap.cc:492] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: No bootstrap required, opened a new log
I20260808 11:48:56.149747 10284 ts_tablet_manager.cc:1397] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 11:48:56.151472 10285 tablet_bootstrap.cc:492] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: No bootstrap required, opened a new log
I20260808 11:48:56.151716 10285 ts_tablet_manager.cc:1397] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260808 11:48:56.152750 10286 tablet_bootstrap.cc:492] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: No bootstrap required, opened a new log
I20260808 11:48:56.152999 10286 ts_tablet_manager.cc:1397] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 11:48:56.159577 10285 raft_consensus.cc:348] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.159986 10285 raft_consensus.cc:374] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:56.160108 10285 raft_consensus.cc:729] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42b499269c45c29870774d96fb9405, State: Initialized, Role: FOLLOWER
I20260808 11:48:56.160482 10284 raft_consensus.cc:348] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.160746 10285 consensus_queue.cc:260] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.160934 10284 raft_consensus.cc:374] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:56.161057 10284 raft_consensus.cc:729] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3247e2a6be4468fb33118a232860fff, State: Initialized, Role: FOLLOWER
I20260808 11:48:56.161623 10284 consensus_queue.cc:260] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.162935 10054 heartbeater.cc:503] Master 127.13.158.126:44439 was elected leader, sending a full tablet report...
I20260808 11:48:56.163488 10284 ts_tablet_manager.cc:1428] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260808 11:48:56.163702 10286 raft_consensus.cc:348] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.164227 10286 raft_consensus.cc:374] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:56.164382 10286 raft_consensus.cc:729] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 970a191ac026492ba041bee75db7ba40, State: Initialized, Role: FOLLOWER
I20260808 11:48:56.165171 10285 ts_tablet_manager.cc:1428] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260808 11:48:56.165098 10286 consensus_queue.cc:260] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.166296 10255 heartbeater.cc:503] Master 127.13.158.126:44439 was elected leader, sending a full tablet report...
I20260808 11:48:56.166991 10286 ts_tablet_manager.cc:1428] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
W20260808 11:48:56.198396 9865 tablet.cc:2369] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:48:56.248049 10056 tablet.cc:2369] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:48:56.292369 10257 tablet.cc:2369] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:56.327834 10297 raft_consensus.cc:482] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:56.328215 10297 raft_consensus.cc:504] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.329929 10297 leader_election.cc:288] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a42b499269c45c29870774d96fb9405 (127.13.158.65:38395), b3247e2a6be4468fb33118a232860fff (127.13.158.66:39251)
I20260808 11:48:56.336229 10280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.330475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:59339 (local address 127.13.158.65:38395)
0808 11:48:56.330741 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.330750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.330775 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:48:56.330977 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.330984 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.331075 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.331212 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:56.331695 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.332538 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.334625 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.335388 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.335520 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.335571 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.335946 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.336037 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:48:56.336311 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.330388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:38395 (local address 127.13.158.67:59339)
0808 11:48:56.330619 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:56.330636 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:56.330808 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:56.331235 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:56.331242 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:56.331276 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:56.331528 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.331546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.332675 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.332683 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.334463 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.334478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.335567 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.335572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.335788 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.335824 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:48:56.335924 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 11:48:56.336056 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 11:48:56.337316 9799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deca623ff9b64c9f8784888db2e6b9c6" candidate_uuid: "970a191ac026492ba041bee75db7ba40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a42b499269c45c29870774d96fb9405" is_pre_election: true
I20260808 11:48:56.337622 10283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.331022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:35035 (local address 127.13.158.66:39251)
0808 11:48:56.331225 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.331231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.331855 (+ 624us) server_negotiation.cc:407] Connection header received
0808 11:48:56.332019 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.332022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.332081 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.332181 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:56.332737 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.333531 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.335521 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.336738 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.336898 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.336969 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.337246 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.337373 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:48:56.337669 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.330890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:39251 (local address 127.13.158.67:35035)
0808 11:48:56.331763 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:56.331776 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:56.331896 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:56.332198 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:56.332202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:56.332225 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:56.332550 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.332559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.333672 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.333677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.335371 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:56.335383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.336927 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.336933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:56.337092 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.337115 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:48:56.337214 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 11:48:56.337334 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":512,"threads_started":1}
I20260808 11:48:56.337909 9799 raft_consensus.cc:2454] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 970a191ac026492ba041bee75db7ba40 in term 0.
I20260808 11:48:56.338627 10084 leader_election.cc:302] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42b499269c45c29870774d96fb9405, 970a191ac026492ba041bee75db7ba40; no voters:
I20260808 11:48:56.338590 9991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deca623ff9b64c9f8784888db2e6b9c6" candidate_uuid: "970a191ac026492ba041bee75db7ba40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3247e2a6be4468fb33118a232860fff" is_pre_election: true
I20260808 11:48:56.339049 10297 raft_consensus.cc:2781] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:48:56.339082 9991 raft_consensus.cc:2454] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 970a191ac026492ba041bee75db7ba40 in term 0.
I20260808 11:48:56.339193 10297 raft_consensus.cc:482] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:48:56.339306 10297 raft_consensus.cc:3037] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:56.341591 10297 raft_consensus.cc:504] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.342288 10297 leader_election.cc:288] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [CANDIDATE]: Term 1 election: Requested vote from peers 8a42b499269c45c29870774d96fb9405 (127.13.158.65:38395), b3247e2a6be4468fb33118a232860fff (127.13.158.66:39251)
I20260808 11:48:56.342619 9799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deca623ff9b64c9f8784888db2e6b9c6" candidate_uuid: "970a191ac026492ba041bee75db7ba40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a42b499269c45c29870774d96fb9405"
I20260808 11:48:56.342758 9991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deca623ff9b64c9f8784888db2e6b9c6" candidate_uuid: "970a191ac026492ba041bee75db7ba40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3247e2a6be4468fb33118a232860fff"
I20260808 11:48:56.342845 9799 raft_consensus.cc:3037] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:56.342972 9991 raft_consensus.cc:3037] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:56.345055 9991 raft_consensus.cc:2454] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 970a191ac026492ba041bee75db7ba40 in term 1.
I20260808 11:48:56.345062 9799 raft_consensus.cc:2454] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 970a191ac026492ba041bee75db7ba40 in term 1.
I20260808 11:48:56.345465 10084 leader_election.cc:302] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42b499269c45c29870774d96fb9405, 970a191ac026492ba041bee75db7ba40; no voters:
I20260808 11:48:56.345777 10297 raft_consensus.cc:2781] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:56.346216 10297 raft_consensus.cc:686] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 1 LEADER]: Becoming Leader. State: Replica: 970a191ac026492ba041bee75db7ba40, State: Running, Role: LEADER
I20260808 11:48:56.346678 10297 consensus_queue.cc:237] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } }
I20260808 11:48:56.351486 9587 catalog_manager.cc:5495] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 970a191ac026492ba041bee75db7ba40 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "970a191ac026492ba041bee75db7ba40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "970a191ac026492ba041bee75db7ba40" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 38369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260808 11:48:56.411124 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.406149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.13.158.67:38369)
0808 11:48:56.406408 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.406413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.406431 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:48:56.406602 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.406607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.406689 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.406882 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:48:56.407203 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.407908 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.409166 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.409537 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.409625 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.410043 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:48:56.410539 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:48:56.410646 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.410765 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.410831 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 11:48:56.419278 9799 raft_consensus.cc:1261] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 1 FOLLOWER]: Refusing update from remote peer 970a191ac026492ba041bee75db7ba40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:56.419271 9991 raft_consensus.cc:1261] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 1 FOLLOWER]: Refusing update from remote peer 970a191ac026492ba041bee75db7ba40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:56.420231 10297 consensus_queue.cc:1035] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42b499269c45c29870774d96fb9405" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:56.420526 10300 consensus_queue.cc:1035] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3247e2a6be4468fb33118a232860fff" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:56.429554 10305 mvcc.cc:205] Tried to move back new op lower bound from 7316233160362557440 to 7316233160079958016. Current Snapshot: MvccSnapshot[applied={T|T < 7316233160362557440}]
I20260808 11:48:56.435521 10307 mvcc.cc:205] Tried to move back new op lower bound from 7316233160362557440 to 7316233160079958016. Current Snapshot: MvccSnapshot[applied={T|T < 7316233160362557440}]
I20260808 11:48:56.445034 9587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:56.445271 9587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:56.448694 9587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 064fc87452304e84a4d74ad520fab002: Sending DeleteTablet for 3 replicas of tablet deca623ff9b64c9f8784888db2e6b9c6
I20260808 11:48:56.449856 9769 tablet_service.cc:1514] Processing DeleteTablet for tablet deca623ff9b64c9f8784888db2e6b9c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:56 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:53812
I20260808 11:48:56.450376 10157 tablet_service.cc:1514] Processing DeleteTablet for tablet deca623ff9b64c9f8784888db2e6b9c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:56 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:45730
I20260808 11:48:56.450340 9969 tablet_service.cc:1514] Processing DeleteTablet for tablet deca623ff9b64c9f8784888db2e6b9c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:56 UTC) from {username='kudu', principal='kudu/127.13.158.126@KRBTEST.COM'} at 127.0.0.1:54560
I20260808 11:48:56.452570 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 9680
I20260808 11:48:56.465709 10321 tablet_replica.cc:330] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: stopping tablet replica
I20260808 11:48:56.466184 10321 raft_consensus.cc:2229] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:48:56.467190 10321 raft_consensus.cc:2258] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:56.469558 10321 ts_tablet_manager.cc:1905] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:56.477934 10321 ts_tablet_manager.cc:1918] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:56.478161 10321 log.cc:1196] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/deca623ff9b64c9f8784888db2e6b9c6
I20260808 11:48:56.478842 10324 tablet_replica.cc:330] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: stopping tablet replica
I20260808 11:48:56.478869 10321 ts_tablet_manager.cc:1939] T deca623ff9b64c9f8784888db2e6b9c6 P 970a191ac026492ba041bee75db7ba40: Deleting consensus metadata
I20260808 11:48:56.481067 10320 tablet_replica.cc:330] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: stopping tablet replica
I20260808 11:48:56.481406 9567 catalog_manager.cc:4841] TS 970a191ac026492ba041bee75db7ba40 (127.13.158.67:38369): tablet deca623ff9b64c9f8784888db2e6b9c6 (table test-table [id=8fee1d43336d4943950286b2eef78fdf]) successfully deleted
I20260808 11:48:56.481473 10324 raft_consensus.cc:2229] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:56.481534 10320 raft_consensus.cc:2229] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:56.482021 10320 raft_consensus.cc:2258] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:56.482031 10324 raft_consensus.cc:2258] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:56.484211 10324 ts_tablet_manager.cc:1905] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:56.484412 10320 ts_tablet_manager.cc:1905] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 11:48:56 dist-test-slave-39l8 krb5kdc[9518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.65@KRBTEST.COM
I20260808 11:48:56.492028 10324 ts_tablet_manager.cc:1918] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:56.492168 10320 ts_tablet_manager.cc:1918] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:56.492430 10320 log.cc:1196] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/deca623ff9b64c9f8784888db2e6b9c6
I20260808 11:48:56.492415 10324 log.cc:1196] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/deca623ff9b64c9f8784888db2e6b9c6
I20260808 11:48:56.493346 10320 ts_tablet_manager.cc:1939] T deca623ff9b64c9f8784888db2e6b9c6 P 8a42b499269c45c29870774d96fb9405: Deleting consensus metadata
I20260808 11:48:56.493556 10324 ts_tablet_manager.cc:1939] T deca623ff9b64c9f8784888db2e6b9c6 P b3247e2a6be4468fb33118a232860fff: Deleting consensus metadata
I20260808 11:48:56.496523 9564 catalog_manager.cc:4841] TS b3247e2a6be4468fb33118a232860fff (127.13.158.66:39251): tablet deca623ff9b64c9f8784888db2e6b9c6 (table test-table [id=8fee1d43336d4943950286b2eef78fdf]) successfully deleted
I20260808 11:48:56.499346 9564 catalog_manager.cc:4841] TS 8a42b499269c45c29870774d96fb9405 (127.13.158.65:38395): tablet deca623ff9b64c9f8784888db2e6b9c6 (table test-table [id=8fee1d43336d4943950286b2eef78fdf]) successfully deleted
I20260808 11:48:56.501216 10280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.454208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.13.158.65:38395)
0808 11:48:56.454431 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.454440 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.475511 (+ 21071us) server_negotiation.cc:407] Connection header received
0808 11:48:56.475896 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.475906 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.476020 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.476111 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:56.479508 (+ 3397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.481081 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.483562 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.484195 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.484300 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.489978 (+ 5678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:56.489991 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:56.490008 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:56.490055 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:56.496020 (+ 5965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.497235 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.497247 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.497253 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.497524 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.498296 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.498303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.498307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.499362 (+ 1055us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:56.499885 (+ 523us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.500301 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.500524 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:48:56.502063 9853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:56.602631 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 9680
I20260808 11:48:56.634570 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 9872
Aug 08 11:48:56 dist-test-slave-39l8 krb5kdc[9518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.66@KRBTEST.COM
I20260808 11:48:56.652766 10283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.635553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54574 (local address 127.13.158.66:39251)
0808 11:48:56.635765 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.635772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.635795 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:48:56.635899 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.635906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.636004 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.636122 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:56.637382 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.638601 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.639904 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.640636 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.640810 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.645376 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:56.645397 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:56.645419 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:56.645463 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:56.649865 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.650544 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.650555 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.650559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.650749 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.651115 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.651121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.651124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.651660 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:56.651963 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.652145 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.652378 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:48:56.653584 10048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:56.746459 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 9872
I20260808 11:48:56.773500 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 10061
Aug 08 11:48:56 dist-test-slave-39l8 krb5kdc[9518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.67@KRBTEST.COM
I20260808 11:48:56.791092 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:56.774298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.13.158.67:38369)
0808 11:48:56.774469 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:56.774476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:56.774545 (+ 69us) server_negotiation.cc:407] Connection header received
0808 11:48:56.774781 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:56.774787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:56.774863 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:56.774982 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:56.776415 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.777260 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.778536 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:56.779181 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:56.779334 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:56.784655 (+ 5321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:56.784672 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:56.784688 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:48:56.784740 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:56.788011 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.788732 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.788739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.788741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.788932 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:48:56.789335 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:48:56.789340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:48:56.789350 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:48:56.789730 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:56.790055 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:56.790372 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 11:48:56.790608 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:48:56.791806 10248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:56.912936 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 10061
I20260808 11:48:56.940697 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 9538
I20260808 11:48:56.941766 9631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:57.050988 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 9538
2026-08-08T11:48:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4851 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T11:48:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:48:57Z Disabled control of system clock
I20260808 11:48:57.150417 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44723
--webserver_interface=127.13.158.126
--webserver_port=0
--builtin_ntp_servers=127.13.158.84:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:44723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:48:57.490713 10334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:57.491103 10334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:57.491255 10334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:57.498363 10334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:48:57.498512 10334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:57.498605 10334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:57.498721 10334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:48:57.498804 10334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:48:57.507851 10334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:44723
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:57.509150 10334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:57.511018 10334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:57.523326 10342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:57.523274 10343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:57.523243 10345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:57.524308 10334 server_base.cc:1034] running on GCE node
I20260808 11:48:57.525084 10334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:57.526492 10334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:57.527715 10334 hybrid_clock.cc:648] HybridClock initialized: now 1786189737527666 us; error 45 us; skew 500 ppm
I20260808 11:48:57.528075 10334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:57.536003 10334 webserver.cc:460] Webserver started at http://127.13.158.126:32925/ using document root <none> and password file <none>
I20260808 11:48:57.536741 10334 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:57.536871 10334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:57.537214 10334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:57.540182 10334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "d58cb57b79404596a2944bea5baef8e6"
format_stamp: "Formatted at 2026-08-08 11:48:57 on dist-test-slave-39l8"
server_key: "1fca59e04719d0bf40a786c968007f2f"
server_key_iv: "12ff4ae10011029a530871be1b1b49c1"
server_key_version: "encryptionkey@0"
I20260808 11:48:57.541011 10334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "d58cb57b79404596a2944bea5baef8e6"
format_stamp: "Formatted at 2026-08-08 11:48:57 on dist-test-slave-39l8"
server_key: "1fca59e04719d0bf40a786c968007f2f"
server_key_iv: "12ff4ae10011029a530871be1b1b49c1"
server_key_version: "encryptionkey@0"
I20260808 11:48:57.546061 10334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 11:48:57.549461 10357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:57.550815 10334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 11:48:57.550997 10334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "d58cb57b79404596a2944bea5baef8e6"
format_stamp: "Formatted at 2026-08-08 11:48:57 on dist-test-slave-39l8"
server_key: "1fca59e04719d0bf40a786c968007f2f"
server_key_iv: "12ff4ae10011029a530871be1b1b49c1"
server_key_version: "encryptionkey@0"
I20260808 11:48:57.551209 10334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:57.575187 10334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:57.576344 10334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:57.576674 10334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:57.595899 10334 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:44723
I20260808 11:48:57.595918 10428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:44723 every 8 connection(s)
I20260808 11:48:57.597302 10334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:48:57.599036 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 10334
I20260808 11:48:57.599232 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:48:57.599795 13945 external_mini_cluster.cc:1428] Setting key 35e073ca6d33fa956a8dace3422a5505
I20260808 11:48:57.601851 10431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:57.613723 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:57.601765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.13.158.126:44723)
0808 11:48:57.602801 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:57.602811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:57.602851 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:48:57.604396 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:57.604416 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:57.604851 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:57.605159 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:57.605739 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:57.607195 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:57.609082 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:57.609723 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:57.609852 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:57.611194 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:57.611208 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:57.611234 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:57.612022 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:57.612098 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:57.612121 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:57.612220 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:57.612445 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 11:48:57.612767 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":407,"threads_started":1}
I20260808 11:48:57.614773 10431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6: Bootstrap starting.
I20260808 11:48:57.617942 10431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:57.619264 10431 log.cc:824] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:57.622053 10431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6: No bootstrap required, opened a new log
I20260808 11:48:57.629177 10431 raft_consensus.cc:348] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58cb57b79404596a2944bea5baef8e6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44723 } }
I20260808 11:48:57.629495 10431 raft_consensus.cc:374] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:57.629580 10431 raft_consensus.cc:729] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58cb57b79404596a2944bea5baef8e6, State: Initialized, Role: FOLLOWER
I20260808 11:48:57.630141 10431 consensus_queue.cc:260] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58cb57b79404596a2944bea5baef8e6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44723 } }
I20260808 11:48:57.630368 10431 raft_consensus.cc:388] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:48:57.630514 10431 raft_consensus.cc:482] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:48:57.630682 10431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:57.633231 10431 raft_consensus.cc:504] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58cb57b79404596a2944bea5baef8e6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44723 } }
I20260808 11:48:57.633713 10431 leader_election.cc:302] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d58cb57b79404596a2944bea5baef8e6; no voters:
I20260808 11:48:57.634269 10431 leader_election.cc:288] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:48:57.634419 10437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:57.635233 10437 raft_consensus.cc:686] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [term 1 LEADER]: Becoming Leader. State: Replica: d58cb57b79404596a2944bea5baef8e6, State: Running, Role: LEADER
I20260808 11:48:57.635800 10437 consensus_queue.cc:237] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58cb57b79404596a2944bea5baef8e6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44723 } }
I20260808 11:48:57.636359 10431 sys_catalog.cc:564] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:48:57.638330 10438 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d58cb57b79404596a2944bea5baef8e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58cb57b79404596a2944bea5baef8e6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44723 } } }
I20260808 11:48:57.638746 10438 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:57.639348 10440 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58cb57b79404596a2944bea5baef8e6. Latest consensus state: current_term: 1 leader_uuid: "d58cb57b79404596a2944bea5baef8e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58cb57b79404596a2944bea5baef8e6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44723 } } }
I20260808 11:48:57.639783 10440 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:48:57.641253 10446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:48:57.647128 10446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:48:57.656034 10446 catalog_manager.cc:1347] Generated new cluster ID: 313c59f9f5044c0485093b4cc1a953c6
I20260808 11:48:57.656160 10446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:48:57.674048 10446 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:48:57.675936 10446 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:48:57.689514 10446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6: Generated new TSK 0
I20260808 11:48:57.690634 10446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:48:57.695622 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.65:0
--local_ip_for_outbound_sockets=127.13.158.65
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44723
--builtin_ntp_servers=127.13.158.84:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:48:58.047617 10472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:58.047881 10472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:58.047964 10472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:58.054966 10472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:58.055063 10472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:58.055214 10472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.65
I20260808 11:48:58.064040 10472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.158.65
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.65
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:58.065225 10472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:58.067396 10472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:58.080447 10483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:58.080435 10486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:58.080689 10484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:58.081372 10472 server_base.cc:1034] running on GCE node
I20260808 11:48:58.082007 10472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:58.082983 10472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:58.084220 10472 hybrid_clock.cc:648] HybridClock initialized: now 1786189738084171 us; error 47 us; skew 500 ppm
I20260808 11:48:58.084547 10472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:58.086828 10472 webserver.cc:460] Webserver started at http://127.13.158.65:41621/ using document root <none> and password file <none>
I20260808 11:48:58.087514 10472 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:58.087639 10472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:58.087986 10472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:58.090855 10472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/data/instance:
uuid: "231200c200c74d6081b6b0c4e908a10a"
format_stamp: "Formatted at 2026-08-08 11:48:58 on dist-test-slave-39l8"
server_key: "4687b17c39e8394b0f2f8e5cdb380154"
server_key_iv: "97e28002dc941a96a44a395d76379607"
server_key_version: "encryptionkey@0"
I20260808 11:48:58.091665 10472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance:
uuid: "231200c200c74d6081b6b0c4e908a10a"
format_stamp: "Formatted at 2026-08-08 11:48:58 on dist-test-slave-39l8"
server_key: "4687b17c39e8394b0f2f8e5cdb380154"
server_key_iv: "97e28002dc941a96a44a395d76379607"
server_key_version: "encryptionkey@0"
I20260808 11:48:58.096561 10472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260808 11:48:58.100350 10499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:58.101683 10472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:48:58.101966 10472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/wal
uuid: "231200c200c74d6081b6b0c4e908a10a"
format_stamp: "Formatted at 2026-08-08 11:48:58 on dist-test-slave-39l8"
server_key: "4687b17c39e8394b0f2f8e5cdb380154"
server_key_iv: "97e28002dc941a96a44a395d76379607"
server_key_version: "encryptionkey@0"
I20260808 11:48:58.102201 10472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:58.117537 10472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:58.118400 10472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:58.118784 10472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:58.119777 10472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:58.121383 10472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:58.121505 10472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:58.121634 10472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:58.121728 10472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:58.155700 10472 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.65:39403
I20260808 11:48:58.155823 10667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.65:39403 every 8 connection(s)
I20260808 11:48:58.157084 10472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/data/info.pb
I20260808 11:48:58.159480 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 10472
I20260808 11:48:58.159693 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/wal/instance
I20260808 11:48:58.160246 13945 external_mini_cluster.cc:1428] Setting key 6cad9b5613c213612505a476f1122b7e
I20260808 11:48:58.163381 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.66:0
--local_ip_for_outbound_sockets=127.13.158.66
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44723
--builtin_ntp_servers=127.13.158.84:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:48:58.169858 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:58.159704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.65:54473 (local address 127.13.158.126:44723)
0808 11:48:58.160236 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:58.160243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:58.161530 (+ 1287us) server_negotiation.cc:407] Connection header received
0808 11:48:58.163467 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:58.163472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:58.163575 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:58.163693 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:58.164742 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:58.165466 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:58.167048 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:58.167463 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:58.167595 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:58.168977 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:58.168987 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:58.168990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:58.169009 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:58.169050 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:58.169063 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:58.169151 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:58.169411 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 11:48:58.169508 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":264,"threads_started":1}
I20260808 11:48:58.170881 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:58.160260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44723 (local address 127.13.158.65:54473)
0808 11:48:58.161355 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:58.161399 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:58.163206 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:58.163929 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:58.163937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:58.164021 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:58.164572 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:58.164589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:58.165618 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:58.165623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:58.166910 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:58.166923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:58.167684 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:58.167693 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:58.167943 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:58.168646 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:58.168649 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:58.168695 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:58.168776 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:58.169167 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:58.169185 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:58.169192 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 11:48:58.169397 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 11:48:58.169798 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":386,"threads_started":1}
I20260808 11:48:58.172433 10668 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44723
I20260808 11:48:58.172746 10668 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:58.173705 10668 heartbeater.cc:511] Master 127.13.158.126:44723 requested a full tablet report, sending...
I20260808 11:48:58.176811 10383 ts_manager.cc:194] Registered new tserver with Master: 231200c200c74d6081b6b0c4e908a10a (127.13.158.65:39403)
I20260808 11:48:58.178846 10383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.65:54473
W20260808 11:48:58.511822 10673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:58.512113 10673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:58.512205 10673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:58.519471 10673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:58.519596 10673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:58.519760 10673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.66
I20260808 11:48:58.528733 10673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.158.66
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.66
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:58.529917 10673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:58.531720 10673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:58.544857 10684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:58.544785 10686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:58.545038 10689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:58.545926 10673 server_base.cc:1034] running on GCE node
I20260808 11:48:58.546525 10673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:58.547535 10673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:58.548754 10673 hybrid_clock.cc:648] HybridClock initialized: now 1786189738548689 us; error 50 us; skew 500 ppm
I20260808 11:48:58.549101 10673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:58.551431 10673 webserver.cc:460] Webserver started at http://127.13.158.66:39535/ using document root <none> and password file <none>
I20260808 11:48:58.552136 10673 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:58.552254 10673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:58.552572 10673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:58.555523 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/data/instance:
uuid: "b9be91b5def44d6791b6d0670eb3a418"
format_stamp: "Formatted at 2026-08-08 11:48:58 on dist-test-slave-39l8"
server_key: "24472f4ff0f510d7b1df7bbfdc75b66d"
server_key_iv: "3d1952ea348e830f29663e388f6e5273"
server_key_version: "encryptionkey@0"
I20260808 11:48:58.556334 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance:
uuid: "b9be91b5def44d6791b6d0670eb3a418"
format_stamp: "Formatted at 2026-08-08 11:48:58 on dist-test-slave-39l8"
server_key: "24472f4ff0f510d7b1df7bbfdc75b66d"
server_key_iv: "3d1952ea348e830f29663e388f6e5273"
server_key_version: "encryptionkey@0"
I20260808 11:48:58.561398 10673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:48:58.565090 10696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:58.566430 10673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:48:58.566630 10673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/wal
uuid: "b9be91b5def44d6791b6d0670eb3a418"
format_stamp: "Formatted at 2026-08-08 11:48:58 on dist-test-slave-39l8"
server_key: "24472f4ff0f510d7b1df7bbfdc75b66d"
server_key_iv: "3d1952ea348e830f29663e388f6e5273"
server_key_version: "encryptionkey@0"
I20260808 11:48:58.566944 10673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:58.586560 10673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:58.587453 10673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:58.587772 10673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:58.588899 10673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:58.590453 10673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:58.590582 10673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:58.590751 10673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:58.590868 10673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 11:48:58.632396 10673 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.66:41897
I20260808 11:48:58.632452 10860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.66:41897 every 8 connection(s)
I20260808 11:48:58.633818 10673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/data/info.pb
I20260808 11:48:58.640496 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 10673
I20260808 11:48:58.640700 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/wal/instance
I20260808 11:48:58.641129 13945 external_mini_cluster.cc:1428] Setting key 0e6d0565dadf3afd9bf55195f65f9c47
I20260808 11:48:58.644023 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.158.67:0
--local_ip_for_outbound_sockets=127.13.158.67
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44723
--builtin_ntp_servers=127.13.158.84:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:48:58.646996 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:58.636403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.66:40499 (local address 127.13.158.126:44723)
0808 11:48:58.636622 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:58.636634 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:58.638043 (+ 1409us) server_negotiation.cc:407] Connection header received
0808 11:48:58.639969 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:58.639975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:58.640078 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:58.640196 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:58.641393 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:58.642360 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:58.644118 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:58.644610 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:58.644750 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:58.646146 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:58.646153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:58.646162 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:58.646182 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:58.646216 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:58.646229 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:58.646324 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:58.646547 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 11:48:58.646656 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:48:58.648171 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:58.636944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44723 (local address 127.13.158.66:40499)
0808 11:48:58.637884 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:58.637935 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:58.639691 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:58.640431 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:58.640439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:58.640526 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:58.641211 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:58.641241 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:58.642504 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:58.642510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:58.643964 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:58.643977 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:58.644841 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:58.644849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:58.645078 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:58.645808 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:58.645811 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:58.645851 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:58.645942 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:58.646326 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:58.646340 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:58.646350 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 11:48:58.646535 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 11:48:58.646987 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":262,"threads_started":1}
I20260808 11:48:58.649536 10861 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44723
I20260808 11:48:58.649869 10861 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:58.650812 10861 heartbeater.cc:511] Master 127.13.158.126:44723 requested a full tablet report, sending...
I20260808 11:48:58.652415 10383 ts_manager.cc:194] Registered new tserver with Master: b9be91b5def44d6791b6d0670eb3a418 (127.13.158.66:41897)
I20260808 11:48:58.653232 10383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.66:40499
W20260808 11:48:58.989799 10864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:48:58.990069 10864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:48:58.990156 10864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:48:58.997193 10864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:48:58.997289 10864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:48:58.997392 10864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.158.67
I20260808 11:48:59.006314 10864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.158.67
--webserver_port=0
--tserver_master_addrs=127.13.158.126:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.158.67
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:48:59.007674 10864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:48:59.009438 10864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:48:59.022434 10872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:59.022578 10877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:48:59.022620 10875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:48:59.023622 10864 server_base.cc:1034] running on GCE node
I20260808 11:48:59.024183 10864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:48:59.025095 10864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:48:59.026301 10864 hybrid_clock.cc:648] HybridClock initialized: now 1786189739026249 us; error 41 us; skew 500 ppm
I20260808 11:48:59.026621 10864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:48:59.028920 10864 webserver.cc:460] Webserver started at http://127.13.158.67:42819/ using document root <none> and password file <none>
I20260808 11:48:59.029633 10864 fs_manager.cc:359] Metadata directory not provided
I20260808 11:48:59.029749 10864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:48:59.030083 10864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:48:59.032979 10864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/data/instance:
uuid: "82b2eb3e09364484859fc0d3d3532d01"
format_stamp: "Formatted at 2026-08-08 11:48:59 on dist-test-slave-39l8"
server_key: "69455f7ca857ef784b67d98ffdd7a0b0"
server_key_iv: "7d56d44ca6facf38cd2ec1952d426fa2"
server_key_version: "encryptionkey@0"
I20260808 11:48:59.033768 10864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance:
uuid: "82b2eb3e09364484859fc0d3d3532d01"
format_stamp: "Formatted at 2026-08-08 11:48:59 on dist-test-slave-39l8"
server_key: "69455f7ca857ef784b67d98ffdd7a0b0"
server_key_iv: "7d56d44ca6facf38cd2ec1952d426fa2"
server_key_version: "encryptionkey@0"
I20260808 11:48:59.039001 10864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:48:59.042349 10886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:59.043653 10864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:48:59.043838 10864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/wal
uuid: "82b2eb3e09364484859fc0d3d3532d01"
format_stamp: "Formatted at 2026-08-08 11:48:59 on dist-test-slave-39l8"
server_key: "69455f7ca857ef784b67d98ffdd7a0b0"
server_key_iv: "7d56d44ca6facf38cd2ec1952d426fa2"
server_key_version: "encryptionkey@0"
I20260808 11:48:59.044075 10864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:48:59.069360 10864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:48:59.070214 10864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:48:59.070574 10864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:48:59.071645 10864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:48:59.073122 10864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:48:59.073244 10864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:59.073382 10864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:48:59.073470 10864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:48:59.106683 10864 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.67:34403
I20260808 11:48:59.106834 11050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.67:34403 every 8 connection(s)
I20260808 11:48:59.108126 10864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/data/info.pb
I20260808 11:48:59.118319 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 10864
I20260808 11:48:59.118516 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/wal/instance
I20260808 11:48:59.119153 13945 external_mini_cluster.cc:1428] Setting key 436f7556827dc552614df3a5d7fd8a9a
I20260808 11:48:59.120788 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.110779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:40151 (local address 127.13.158.126:44723)
0808 11:48:59.111005 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.111012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.112394 (+ 1382us) server_negotiation.cc:407] Connection header received
0808 11:48:59.114115 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.114119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.114189 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.114279 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:59.115377 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.116130 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.117585 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.117995 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.118091 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.119693 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:59.119700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:59.119706 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:59.119728 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:59.119769 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:59.119784 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:59.119872 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.120343 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.120437 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:48:59.122047 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.111326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44723 (local address 127.13.158.67:40151)
0808 11:48:59.112198 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:59.112255 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:59.113861 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:59.114477 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:59.114484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:59.114567 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:48:59.115194 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.115233 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.116265 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.116269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.117457 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.117468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.118237 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.118244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.118562 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.119322 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:48:59.119325 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:48:59.119374 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:48:59.119473 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:48:59.119888 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:48:59.119899 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:48:59.119907 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:48:59.120106 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 11:48:59.120408 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":268,"threads_started":1}
I20260808 11:48:59.123421 11051 heartbeater.cc:347] Connected to a master server at 127.13.158.126:44723
I20260808 11:48:59.123672 11051 heartbeater.cc:464] Registering TS with master...
I20260808 11:48:59.124337 11051 heartbeater.cc:511] Master 127.13.158.126:44723 requested a full tablet report, sending...
I20260808 11:48:59.125607 10383 ts_manager.cc:194] Registered new tserver with Master: 82b2eb3e09364484859fc0d3d3532d01 (127.13.158.67:34403)
I20260808 11:48:59.126385 10383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.158.67:40151
I20260808 11:48:59.134557 13945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:48:59.144835 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.139124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39296 (local address 127.13.158.126:44723)
0808 11:48:59.139285 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.139290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.139780 (+ 490us) server_negotiation.cc:407] Connection header received
0808 11:48:59.139976 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.139985 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.140056 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.140152 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:59.140621 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.141288 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.142607 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.143103 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.143240 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.144170 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:59.144180 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:59.144185 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:59.144201 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:59.144236 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:59.144247 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:59.144337 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.144473 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.144565 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:48:59.148336 10383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:48:59.155010 10383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:48:59.178802 11066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.167467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41897 (local address 127.0.0.1:51676)
0808 11:48:59.170146 (+ 2679us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:59.170167 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:59.170352 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:59.171275 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:59.171279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:59.171302 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:59.171728 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.171744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.173622 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.173629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.175522 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.175539 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.177417 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.177424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.177898 (+ 474us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.177957 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 11:48:59.178086 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 11:48:59.178292 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2544,"mutex_wait_us":40,"thread_start_us":535,"threads_started":1}
I20260808 11:48:59.178844 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.166905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45376 (local address 127.13.158.67:34403)
0808 11:48:59.169481 (+ 2576us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.169492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.169528 (+ 36us) server_negotiation.cc:407] Connection header received
0808 11:48:59.169633 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.169639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.169914 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.170964 (+ 1050us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:59.171265 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.172777 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.175812 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.177229 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.177368 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.177653 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.178264 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.178438 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2406,"thread_start_us":2257,"threads_started":1}
I20260808 11:48:59.179459 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.166557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.67:34403 (local address 127.0.0.1:45376)
0808 11:48:59.168540 (+ 1983us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:59.168608 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:59.168820 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:59.170151 (+ 1331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:59.170160 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:59.170197 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:59.170750 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.170764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.172896 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.172903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.175561 (+ 2658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.175573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.177794 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.177801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.178005 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.178034 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 11:48:59.178111 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 11:48:59.179109 (+ 998us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1620,"mutex_wait_us":859,"thread_start_us":3323,"threads_started":1}
I20260808 11:48:59.179742 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.170229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39403 (local address 127.0.0.1:54328)
0808 11:48:59.172242 (+ 2013us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:59.172263 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:59.172376 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:59.172960 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:59.172967 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:59.173000 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:59.173464 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.173476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.175351 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.175356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.176777 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.176790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.178661 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.178668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.178889 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.179169 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 11:48:59.179266 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 11:48:59.179391 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1910,"thread_start_us":1782,"threads_started":1}
I20260808 11:48:59.179755 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.167620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51676 (local address 127.13.158.66:41897)
0808 11:48:59.169756 (+ 2136us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.169764 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.170249 (+ 485us) server_negotiation.cc:407] Connection header received
0808 11:48:59.170600 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.170607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.170849 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.170992 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:59.171930 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.173449 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.175740 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.177232 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.177368 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.177625 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.178610 (+ 985us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.178795 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1977,"thread_start_us":1821,"threads_started":1}
I20260808 11:48:59.180789 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.170822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54328 (local address 127.13.158.65:39403)
0808 11:48:59.171380 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.171391 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.172366 (+ 975us) server_negotiation.cc:407] Connection header received
0808 11:48:59.172535 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.172543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.172751 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.172934 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:59.173695 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.175221 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.176962 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.178187 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.178300 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.178527 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.179326 (+ 799us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.179486 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":281,"threads_started":1}
I20260808 11:48:59.181874 10668 heartbeater.cc:503] Master 127.13.158.126:44723 was elected leader, sending a full tablet report...
I20260808 11:48:59.183337 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 66c4d0a9fc1c46e19e29e4a6560ca124 (DEFAULT_TABLE table=test-table [id=6dcf996eedd5446083ed22551c8fca72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:59.183353 10765 tablet_service.cc:1467] Processing CreateTablet for tablet 66c4d0a9fc1c46e19e29e4a6560ca124 (DEFAULT_TABLE table=test-table [id=6dcf996eedd5446083ed22551c8fca72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:59.184520 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66c4d0a9fc1c46e19e29e4a6560ca124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:59.184079 10576 tablet_service.cc:1467] Processing CreateTablet for tablet 66c4d0a9fc1c46e19e29e4a6560ca124 (DEFAULT_TABLE table=test-table [id=6dcf996eedd5446083ed22551c8fca72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:48:59.184670 10765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66c4d0a9fc1c46e19e29e4a6560ca124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:59.185297 10576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66c4d0a9fc1c46e19e29e4a6560ca124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:48:59.195681 11073 tablet_bootstrap.cc:492] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Bootstrap starting.
I20260808 11:48:59.198839 11073 tablet_bootstrap.cc:654] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:59.199401 11074 tablet_bootstrap.cc:492] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: Bootstrap starting.
I20260808 11:48:59.201038 11076 tablet_bootstrap.cc:492] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Bootstrap starting.
I20260808 11:48:59.201522 11073 log.cc:824] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:59.202911 11074 tablet_bootstrap.cc:654] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:59.204070 11076 tablet_bootstrap.cc:654] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Neither blocks nor log segments found. Creating new log.
I20260808 11:48:59.204061 11074 log.cc:824] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:59.204921 11073 tablet_bootstrap.cc:492] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: No bootstrap required, opened a new log
I20260808 11:48:59.205190 11073 ts_tablet_manager.cc:1397] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260808 11:48:59.205188 11076 log.cc:824] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Log is configured to *not* fsync() on all Append() calls
I20260808 11:48:59.207067 11074 tablet_bootstrap.cc:492] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: No bootstrap required, opened a new log
I20260808 11:48:59.207320 11074 ts_tablet_manager.cc:1397] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260808 11:48:59.207716 11076 tablet_bootstrap.cc:492] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: No bootstrap required, opened a new log
I20260808 11:48:59.207978 11076 ts_tablet_manager.cc:1397] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260808 11:48:59.218006 11073 raft_consensus.cc:348] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.217985 11074 raft_consensus.cc:348] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.218461 11074 raft_consensus.cc:374] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:59.218461 11073 raft_consensus.cc:374] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:59.218571 11074 raft_consensus.cc:729] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 231200c200c74d6081b6b0c4e908a10a, State: Initialized, Role: FOLLOWER
I20260808 11:48:59.218578 11073 raft_consensus.cc:729] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b2eb3e09364484859fc0d3d3532d01, State: Initialized, Role: FOLLOWER
I20260808 11:48:59.218659 11076 raft_consensus.cc:348] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.219113 11076 raft_consensus.cc:374] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:48:59.219225 11076 raft_consensus.cc:729] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9be91b5def44d6791b6d0670eb3a418, State: Initialized, Role: FOLLOWER
I20260808 11:48:59.219383 11074 consensus_queue.cc:260] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.219393 11073 consensus_queue.cc:260] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.219890 11076 consensus_queue.cc:260] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.227928 11074 ts_tablet_manager.cc:1428] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: Time spent starting tablet: real 0.020s user 0.010s sys 0.009s
I20260808 11:48:59.229331 10861 heartbeater.cc:503] Master 127.13.158.126:44723 was elected leader, sending a full tablet report...
I20260808 11:48:59.229326 11051 heartbeater.cc:503] Master 127.13.158.126:44723 was elected leader, sending a full tablet report...
I20260808 11:48:59.230278 11076 ts_tablet_manager.cc:1428] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Time spent starting tablet: real 0.022s user 0.015s sys 0.007s
I20260808 11:48:59.231302 11073 ts_tablet_manager.cc:1428] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Time spent starting tablet: real 0.026s user 0.019s sys 0.005s
I20260808 11:48:59.311616 11081 raft_consensus.cc:482] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:48:59.311923 11081 raft_consensus.cc:504] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.313534 11081 leader_election.cc:288] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 231200c200c74d6081b6b0c4e908a10a (127.13.158.65:39403), b9be91b5def44d6791b6d0670eb3a418 (127.13.158.66:41897)
I20260808 11:48:59.319960 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.314130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:57161 (local address 127.13.158.66:41897)
0808 11:48:59.314342 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.314350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.314372 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:48:59.314472 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.314479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.314571 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.314689 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:59.315216 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.316186 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.318157 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.319134 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.319254 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.319302 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.319631 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.319713 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:48:59.320075 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.313827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.66:41897 (local address 127.13.158.67:57161)
0808 11:48:59.314109 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:59.314135 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:59.314294 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:59.314774 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:59.314779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:59.314830 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:59.315022 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.315036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.316327 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.316332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.317984 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.317996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.319291 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.319297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.319472 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.319510 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 11:48:59.319607 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 11:48:59.319748 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":28}
I20260808 11:48:59.320833 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.313945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.65:39403 (local address 127.13.158.67:46977)
0808 11:48:59.314836 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0808 11:48:59.314860 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:48:59.314978 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:48:59.315339 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:48:59.315343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:48:59.315370 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:48:59.315754 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.315766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.317006 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.317009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.318559 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:48:59.318574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.319952 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.319960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:48:59.320342 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.320391 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 11:48:59.320464 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 11:48:59.320544 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":1,"thread_start_us":474,"threads_started":1}
I20260808 11:48:59.320849 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.314033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.158.67:46977 (local address 127.13.158.65:39403)
0808 11:48:59.314233 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.314240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.314968 (+ 728us) server_negotiation.cc:407] Connection header received
0808 11:48:59.315093 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.315101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.315212 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.315289 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:48:59.315928 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.316854 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.318798 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.319783 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.319933 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.319992 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.320531 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.320657 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:48:59.321274 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66c4d0a9fc1c46e19e29e4a6560ca124" candidate_uuid: "82b2eb3e09364484859fc0d3d3532d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9be91b5def44d6791b6d0670eb3a418" is_pre_election: true
I20260808 11:48:59.321839 10794 raft_consensus.cc:2454] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82b2eb3e09364484859fc0d3d3532d01 in term 0.
I20260808 11:48:59.321807 10606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66c4d0a9fc1c46e19e29e4a6560ca124" candidate_uuid: "82b2eb3e09364484859fc0d3d3532d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "231200c200c74d6081b6b0c4e908a10a" is_pre_election: true
I20260808 11:48:59.322314 10606 raft_consensus.cc:2454] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82b2eb3e09364484859fc0d3d3532d01 in term 0.
I20260808 11:48:59.322516 10890 leader_election.cc:302] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82b2eb3e09364484859fc0d3d3532d01, b9be91b5def44d6791b6d0670eb3a418; no voters:
I20260808 11:48:59.322957 11081 raft_consensus.cc:2781] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:48:59.323113 11081 raft_consensus.cc:482] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:48:59.323220 11081 raft_consensus.cc:3037] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:59.325637 11081 raft_consensus.cc:504] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.326179 11081 leader_election.cc:288] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [CANDIDATE]: Term 1 election: Requested vote from peers 231200c200c74d6081b6b0c4e908a10a (127.13.158.65:39403), b9be91b5def44d6791b6d0670eb3a418 (127.13.158.66:41897)
I20260808 11:48:59.326656 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66c4d0a9fc1c46e19e29e4a6560ca124" candidate_uuid: "82b2eb3e09364484859fc0d3d3532d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9be91b5def44d6791b6d0670eb3a418"
I20260808 11:48:59.326648 10606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66c4d0a9fc1c46e19e29e4a6560ca124" candidate_uuid: "82b2eb3e09364484859fc0d3d3532d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "231200c200c74d6081b6b0c4e908a10a"
I20260808 11:48:59.326875 10606 raft_consensus.cc:3037] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:59.326874 10794 raft_consensus.cc:3037] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:48:59.330130 10794 raft_consensus.cc:2454] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82b2eb3e09364484859fc0d3d3532d01 in term 1.
I20260808 11:48:59.330127 10606 raft_consensus.cc:2454] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82b2eb3e09364484859fc0d3d3532d01 in term 1.
I20260808 11:48:59.330524 10891 leader_election.cc:302] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 231200c200c74d6081b6b0c4e908a10a, 82b2eb3e09364484859fc0d3d3532d01; no voters:
I20260808 11:48:59.330850 11081 raft_consensus.cc:2781] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:48:59.331300 11081 raft_consensus.cc:686] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 1 LEADER]: Becoming Leader. State: Replica: 82b2eb3e09364484859fc0d3d3532d01, State: Running, Role: LEADER
I20260808 11:48:59.331766 11081 consensus_queue.cc:237] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } }
I20260808 11:48:59.336184 10382 catalog_manager.cc:5495] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82b2eb3e09364484859fc0d3d3532d01 (127.13.158.67). New cstate: current_term: 1 leader_uuid: "82b2eb3e09364484859fc0d3d3532d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82b2eb3e09364484859fc0d3d3532d01" member_type: VOTER last_known_addr { host: "127.13.158.67" port: 34403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 } health_report { overall_health: UNKNOWN } } }
W20260808 11:48:59.359900 11053 tablet.cc:2369] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:59.374357 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.368545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45386 (local address 127.13.158.67:34403)
0808 11:48:59.368779 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.368785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.368888 (+ 103us) server_negotiation.cc:407] Connection header received
0808 11:48:59.369123 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.369127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.369206 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.369354 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:48:59.369731 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.370599 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.372242 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.372663 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.372796 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.373289 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:48:59.373786 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:48:59.373943 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.374025 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.374104 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:48:59.382229 10606 raft_consensus.cc:1261] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 1 FOLLOWER]: Refusing update from remote peer 82b2eb3e09364484859fc0d3d3532d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:59.382241 10794 raft_consensus.cc:1261] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 1 FOLLOWER]: Refusing update from remote peer 82b2eb3e09364484859fc0d3d3532d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:48:59.383157 11086 consensus_queue.cc:1035] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "231200c200c74d6081b6b0c4e908a10a" member_type: VOTER last_known_addr { host: "127.13.158.65" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:48:59.383466 11081 consensus_queue.cc:1035] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9be91b5def44d6791b6d0670eb3a418" member_type: VOTER last_known_addr { host: "127.13.158.66" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:48:59.386004 10862 tablet.cc:2369] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:48:59.396780 11089 mvcc.cc:205] Tried to move back new op lower bound from 7316233172500004864 to 7316233172307124224. Current Snapshot: MvccSnapshot[applied={T|T < 7316233172500004864 or (T in {7316233172500004864})}]
I20260808 11:48:59.406226 10382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:59.406518 10382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:48:59.410365 10382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d58cb57b79404596a2944bea5baef8e6: Sending DeleteTablet for 3 replicas of tablet 66c4d0a9fc1c46e19e29e4a6560ca124
I20260808 11:48:59.411455 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 66c4d0a9fc1c46e19e29e4a6560ca124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:59 UTC) from {username='slave'} at 127.0.0.1:45376
I20260808 11:48:59.411564 10576 tablet_service.cc:1514] Processing DeleteTablet for tablet 66c4d0a9fc1c46e19e29e4a6560ca124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:59 UTC) from {username='slave'} at 127.0.0.1:54328
I20260808 11:48:59.411871 10765 tablet_service.cc:1514] Processing DeleteTablet for tablet 66c4d0a9fc1c46e19e29e4a6560ca124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:48:59 UTC) from {username='slave'} at 127.0.0.1:51676
I20260808 11:48:59.412415 11104 tablet_replica.cc:330] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: stopping tablet replica
I20260808 11:48:59.412901 11104 raft_consensus.cc:2229] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:48:59.413619 11104 raft_consensus.cc:2258] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:59.413691 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 10472
I20260808 11:48:59.414393 11105 tablet_replica.cc:330] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: stopping tablet replica
I20260808 11:48:59.414698 11105 raft_consensus.cc:2229] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:59.415198 11105 raft_consensus.cc:2258] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:59.419765 11105 ts_tablet_manager.cc:1905] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:59.423646 11107 tablet_replica.cc:330] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: stopping tablet replica
I20260808 11:48:59.424090 11107 raft_consensus.cc:2229] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:48:59.424655 11107 raft_consensus.cc:2258] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:48:59.424875 11104 ts_tablet_manager.cc:1905] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:59.426749 11107 ts_tablet_manager.cc:1905] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:48:59.427665 11105 ts_tablet_manager.cc:1918] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:59.427832 11105 log.cc:1196] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-0/wal/wals/66c4d0a9fc1c46e19e29e4a6560ca124
I20260808 11:48:59.428361 11105 ts_tablet_manager.cc:1939] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 231200c200c74d6081b6b0c4e908a10a: Deleting consensus metadata
I20260808 11:48:59.430199 10367 catalog_manager.cc:4841] TS 231200c200c74d6081b6b0c4e908a10a (127.13.158.65:39403): tablet 66c4d0a9fc1c46e19e29e4a6560ca124 (table test-table [id=6dcf996eedd5446083ed22551c8fca72]) successfully deleted
I20260808 11:48:59.432379 11104 ts_tablet_manager.cc:1918] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:59.432555 11104 log.cc:1196] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-2/wal/wals/66c4d0a9fc1c46e19e29e4a6560ca124
I20260808 11:48:59.433113 11104 ts_tablet_manager.cc:1939] T 66c4d0a9fc1c46e19e29e4a6560ca124 P 82b2eb3e09364484859fc0d3d3532d01: Deleting consensus metadata
I20260808 11:48:59.434170 11107 ts_tablet_manager.cc:1918] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:48:59.434386 11107 log.cc:1196] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Deleting WAL directory at /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189414476033-13945-0/minicluster-data/ts-1/wal/wals/66c4d0a9fc1c46e19e29e4a6560ca124
I20260808 11:48:59.434783 10367 catalog_manager.cc:4841] TS 82b2eb3e09364484859fc0d3d3532d01 (127.13.158.67:34403): tablet 66c4d0a9fc1c46e19e29e4a6560ca124 (table test-table [id=6dcf996eedd5446083ed22551c8fca72]) successfully deleted
I20260808 11:48:59.435094 11107 ts_tablet_manager.cc:1939] T 66c4d0a9fc1c46e19e29e4a6560ca124 P b9be91b5def44d6791b6d0670eb3a418: Deleting consensus metadata
I20260808 11:48:59.436800 10362 catalog_manager.cc:4841] TS b9be91b5def44d6791b6d0670eb3a418 (127.13.158.66:41897): tablet 66c4d0a9fc1c46e19e29e4a6560ca124 (table test-table [id=6dcf996eedd5446083ed22551c8fca72]) successfully deleted
I20260808 11:48:59.437871 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.414812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.13.158.65:39403)
0808 11:48:59.414985 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.414991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.423249 (+ 8258us) server_negotiation.cc:407] Connection header received
0808 11:48:59.423453 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.423458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.423570 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.423671 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:59.431092 (+ 7421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.432213 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.434226 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.434862 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.434962 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.436322 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:59.436331 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:59.436344 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:59.437064 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:59.437108 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:59.437128 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:59.437249 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.437410 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.437529 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:48:59.438546 10661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:59.548393 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 10472
I20260808 11:48:59.576198 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 10673
I20260808 11:48:59.583951 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.577057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.13.158.66:41897)
0808 11:48:59.577221 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.577226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.577244 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:48:59.577460 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.577465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.577550 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.577629 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:59.578103 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.578894 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.580676 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.581077 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.581168 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.582425 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:59.582434 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:59.582448 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:59.583191 (+ 743us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:59.583243 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:59.583263 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:59.583348 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.583513 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.583596 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:48:59.584708 10854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:59.698879 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 10673
I20260808 11:48:59.728828 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 10864
I20260808 11:48:59.737545 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:48:59.729818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45392 (local address 127.13.158.67:34403)
0808 11:48:59.729977 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 11:48:59.729981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:48:59.729998 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:48:59.730123 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:48:59.730127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:48:59.730240 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:48:59.730329 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:48:59.730841 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.731640 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.733593 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:48:59.734033 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:48:59.734129 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:48:59.735496 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:48:59.735512 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:48:59.735534 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:48:59.736469 (+ 935us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:48:59.736546 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:48:59.736585 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:48:59.736697 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 11:48:59.737140 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 11:48:59.737253 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 11:48:59.738402 11046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:48:59.867851 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 10864
I20260808 11:48:59.894037 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 10334
I20260808 11:48:59.895330 10423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:00.004573 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 10334
2026-08-08T11:49:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2959 ms)
[----------] 6 tests from EncryptionPolicyTest (23347 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:00 dist-test-slave-39l8 krb5kdc[11118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:00 dist-test-slave-39l8 krb5kdc[11118](info): set up 2 sockets
Aug 08 11:49:00 dist-test-slave-39l8 krb5kdc[11118](info): commencing operation
krb5kdc: starting...
W20260808 11:49:01.314082 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:01 dist-test-slave-39l8 krb5kdc[11118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:01.644553 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:01.985749 11141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:01.986095 11141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:01.986271 11141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:01.993678 11141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:01.993821 11141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:01.993934 11141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:01.994074 11141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:01.994185 11141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:02.004667 11141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42693
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:02.005988 11141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:02.007414 11141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:02.021631 11153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:02.021687 11147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:02.022984 11149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:02.024034 11141 server_base.cc:1034] running on GCE node
I20260808 11:49:02.024838 11141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:02.026361 11141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:02.027604 11141 hybrid_clock.cc:648] HybridClock initialized: now 1786189742027556 us; error 59 us; skew 500 ppm
I20260808 11:49:02.027980 11141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:02 dist-test-slave-39l8 krb5kdc[11118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189742, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:02.031104 11141 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:02.032595 11141 webserver.cc:460] Webserver started at http://127.0.0.1:46173/ using document root <none> and password file <none>
I20260808 11:49:02.033336 11141 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:02.033480 11141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:02.033789 11141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:02.036983 11141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "d7aaf6de02894e10851982ea290d74a1"
format_stamp: "Formatted at 2026-08-08 11:49:02 on dist-test-slave-39l8"
server_key: "d196da40bc9034383d252a1e806f3bff"
server_key_iv: "3d2822c1b31d4b39cfbc22536f67d3af"
server_key_version: "encryptionkey@0"
I20260808 11:49:02.037771 11141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "d7aaf6de02894e10851982ea290d74a1"
format_stamp: "Formatted at 2026-08-08 11:49:02 on dist-test-slave-39l8"
server_key: "d196da40bc9034383d252a1e806f3bff"
server_key_iv: "3d2822c1b31d4b39cfbc22536f67d3af"
server_key_version: "encryptionkey@0"
I20260808 11:49:02.042810 11141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 11:49:02.046273 11163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:02.047600 11141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:02.047787 11141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "d7aaf6de02894e10851982ea290d74a1"
format_stamp: "Formatted at 2026-08-08 11:49:02 on dist-test-slave-39l8"
server_key: "d196da40bc9034383d252a1e806f3bff"
server_key_iv: "3d2822c1b31d4b39cfbc22536f67d3af"
server_key_version: "encryptionkey@0"
I20260808 11:49:02.048009 11141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:02.086551 11141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:02.090785 11141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:02.091086 11141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:02.109781 11141 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42693
I20260808 11:49:02.109822 11239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42693 every 8 connection(s)
I20260808 11:49:02.111204 11141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:02.115104 11241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:02.117218 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 11141
I20260808 11:49:02.117468 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:02.118109 13945 external_mini_cluster.cc:1428] Setting key fbbcf06a96ba1e12170f0034aa4511d5
I20260808 11:49:02.128316 11241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1: Bootstrap starting.
I20260808 11:49:02.131826 11241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:02.132839 11241 log.cc:824] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:49:02 dist-test-slave-39l8 krb5kdc[11118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 11:49:02.136039 11241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1: No bootstrap required, opened a new log
I20260808 11:49:02.143230 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:02.120349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34269 (local address 127.0.0.1:42693)
0808 11:49:02.121387 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:02.121402 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:02.121828 (+ 426us) server_negotiation.cc:407] Connection header received
0808 11:49:02.124153 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:02.124186 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:02.124697 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:02.125150 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:02.126510 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:02.128513 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:02.129775 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:02.130828 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:02.131038 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:02.135769 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:02.135776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:02.135790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:02.135827 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:02.139017 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:02.139877 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:02.139882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:02.139887 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:02.140031 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:02.140557 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:02.140560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:02.140562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:02.141066 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:02.141298 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:02.141787 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 11:49:02.142077 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":501,"threads_started":1}
I20260808 11:49:02.144578 11241 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7aaf6de02894e10851982ea290d74a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } }
I20260808 11:49:02.144932 11241 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:02.145047 11241 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7aaf6de02894e10851982ea290d74a1, State: Initialized, Role: FOLLOWER
I20260808 11:49:02.145630 11241 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7aaf6de02894e10851982ea290d74a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } }
I20260808 11:49:02.145902 11241 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:02.146059 11241 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:02.146251 11241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:02.147394 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:02.148679 11241 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7aaf6de02894e10851982ea290d74a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } }
I20260808 11:49:02.149140 11241 leader_election.cc:302] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7aaf6de02894e10851982ea290d74a1; no voters:
I20260808 11:49:02.149713 11241 leader_election.cc:288] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:02.149858 11246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:02.151749 11241 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:02.159704 11246 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [term 1 LEADER]: Becoming Leader. State: Replica: d7aaf6de02894e10851982ea290d74a1, State: Running, Role: LEADER
I20260808 11:49:02.160526 11246 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7aaf6de02894e10851982ea290d74a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } }
I20260808 11:49:02.164706 11250 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7aaf6de02894e10851982ea290d74a1. Latest consensus state: current_term: 1 leader_uuid: "d7aaf6de02894e10851982ea290d74a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7aaf6de02894e10851982ea290d74a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } } }
I20260808 11:49:02.165179 11250 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:02.165869 11248 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7aaf6de02894e10851982ea290d74a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7aaf6de02894e10851982ea290d74a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } } }
I20260808 11:49:02.166239 11248 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1 [sys.catalog]: This master's current role is: LEADER
W20260808 11:49:02.181072 11277 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:49:02.181237 11277 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:49:02.181365 11279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:02.183415 11279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:02.192972 11279 catalog_manager.cc:1347] Generated new cluster ID: 3dc12d2e850c4ba1a6bb880f89df6bf7
I20260808 11:49:02.193163 11279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:02.196446 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:02.164406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41999 (local address 127.0.0.1:42693)
0808 11:49:02.164582 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:02.164590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:02.175115 (+ 10525us) server_negotiation.cc:407] Connection header received
0808 11:49:02.176358 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:02.176368 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:02.176495 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:02.176630 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:02.178951 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:02.180209 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:02.182054 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:02.182477 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:02.182588 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:02.185885 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:02.185892 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:02.185899 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:02.185937 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:02.192827 (+ 6890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:02.193922 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:02.193927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:02.193931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:02.194113 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:02.194751 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:02.194759 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:02.194762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:02.195166 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:02.195439 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:02.195870 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 11:49:02.196067 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:49:02.206799 11279 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:02.208580 11279 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:02.220588 11279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7aaf6de02894e10851982ea290d74a1: Generated new TSK 0
I20260808 11:49:02.221596 11279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:02.236600 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11141
I20260808 11:49:02.237879 11231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:02.329876 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 11141
W20260808 11:49:02.356077 11116 connection.cc:535] client connection to 127.0.0.1:42693 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42693): Connection reset by peer (error 104)
2026-08-08T11:49:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:02 dist-test-slave-39l8 krb5kdc[11290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:02 dist-test-slave-39l8 krb5kdc[11290](info): set up 2 sockets
Aug 08 11:49:02 dist-test-slave-39l8 krb5kdc[11290](info): commencing operation
krb5kdc: starting...
W20260808 11:49:03.622890 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:03 dist-test-slave-39l8 krb5kdc[11290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:03.915730 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36331
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:04.256109 11312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:04.256387 11312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:04.256569 11312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:04.263743 11312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:04.263861 11312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:04.263950 11312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:04.264056 11312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:04.264132 11312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:04.273815 11312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36331
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:04.275139 11312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:04.276415 11312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:04.288019 11318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:04.287963 11322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:04.287914 11319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:04.288676 11312 server_base.cc:1034] running on GCE node
I20260808 11:49:04.289489 11312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:04.290825 11312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:04.292106 11312 hybrid_clock.cc:648] HybridClock initialized: now 1786189744292053 us; error 46 us; skew 500 ppm
I20260808 11:49:04.292454 11312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:04 dist-test-slave-39l8 krb5kdc[11290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189744, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:04.300918 11312 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:04.302361 11312 webserver.cc:460] Webserver started at http://127.0.0.1:36577/ using document root <none> and password file <none>
I20260808 11:49:04.303138 11312 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:04.303294 11312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:04.303604 11312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:04.306655 11312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "5c7bc49d405d4967aac6cf3ea2855574"
format_stamp: "Formatted at 2026-08-08 11:49:04 on dist-test-slave-39l8"
server_key: "08a1862dae03151dca8293c232da053f"
server_key_iv: "418c5207b92a72a745386be738b39093"
server_key_version: "encryptionkey@0"
I20260808 11:49:04.307490 11312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "5c7bc49d405d4967aac6cf3ea2855574"
format_stamp: "Formatted at 2026-08-08 11:49:04 on dist-test-slave-39l8"
server_key: "08a1862dae03151dca8293c232da053f"
server_key_iv: "418c5207b92a72a745386be738b39093"
server_key_version: "encryptionkey@0"
I20260808 11:49:04.312602 11312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:49:04.316021 11331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:04.317269 11312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:49:04.317447 11312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "5c7bc49d405d4967aac6cf3ea2855574"
format_stamp: "Formatted at 2026-08-08 11:49:04 on dist-test-slave-39l8"
server_key: "08a1862dae03151dca8293c232da053f"
server_key_iv: "418c5207b92a72a745386be738b39093"
server_key_version: "encryptionkey@0"
I20260808 11:49:04.317641 11312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:04.346590 11312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:04.350535 11312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:04.350863 11312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:04.369406 11312 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36331
I20260808 11:49:04.369438 11410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36331 every 8 connection(s)
I20260808 11:49:04.370831 11312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:04.374508 11412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:04.376997 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 11312
I20260808 11:49:04.377180 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:04.377611 13945 external_mini_cluster.cc:1428] Setting key 228bac0784293f37e0a8b9e818f02f15
I20260808 11:49:04.387890 11412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574: Bootstrap starting.
I20260808 11:49:04.391914 11412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:04.392885 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:04.379584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44981 (local address 127.0.0.1:36331)
0808 11:49:04.380376 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:04.380396 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:04.380451 (+ 55us) server_negotiation.cc:407] Connection header received
0808 11:49:04.382749 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:04.382776 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:04.383283 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:04.383698 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:04.384232 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:04.386236 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:04.387721 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:04.388689 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:04.388854 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:04.390049 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:04.390060 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:04.390076 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:04.390806 (+ 730us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:04.390911 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:04.390957 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:04.391120 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:04.391322 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 11:49:04.391483 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":307,"threads_started":1}
I20260808 11:49:04.393316 11412 log.cc:824] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:04.396582 11412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574: No bootstrap required, opened a new log
I20260808 11:49:04.397243 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:04.407068 11412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7bc49d405d4967aac6cf3ea2855574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } }
I20260808 11:49:04.407546 11412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:04.407730 11412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7bc49d405d4967aac6cf3ea2855574, State: Initialized, Role: FOLLOWER
I20260808 11:49:04.408450 11412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7bc49d405d4967aac6cf3ea2855574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } }
I20260808 11:49:04.408761 11412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:04.408963 11412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:04.409190 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:04.402272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51061 (local address 127.0.0.1:36331)
0808 11:49:04.402492 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:04.402501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:04.402937 (+ 436us) server_negotiation.cc:407] Connection header received
0808 11:49:04.403116 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:04.403120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:04.403174 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:04.403270 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:04.403867 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:04.404623 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:04.406481 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:04.406965 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:04.407058 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:04.408336 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:04.408341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:04.408346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:04.408364 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:04.408452 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:04.408475 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:04.408598 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:04.408757 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 11:49:04.408823 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:49:04.409159 11412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:04.411633 11412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7bc49d405d4967aac6cf3ea2855574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } }
I20260808 11:49:04.412091 11412 leader_election.cc:302] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c7bc49d405d4967aac6cf3ea2855574; no voters:
I20260808 11:49:04.412626 11412 leader_election.cc:288] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:04.412768 11432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:04.413586 11432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [term 1 LEADER]: Becoming Leader. State: Replica: 5c7bc49d405d4967aac6cf3ea2855574, State: Running, Role: LEADER
I20260808 11:49:04.414189 11432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7bc49d405d4967aac6cf3ea2855574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } }
I20260808 11:49:04.414916 11412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:04.417192 11435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c7bc49d405d4967aac6cf3ea2855574. Latest consensus state: current_term: 1 leader_uuid: "5c7bc49d405d4967aac6cf3ea2855574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7bc49d405d4967aac6cf3ea2855574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } } }
I20260808 11:49:04.417657 11435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:04.418569 11433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c7bc49d405d4967aac6cf3ea2855574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7bc49d405d4967aac6cf3ea2855574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } } }
I20260808 11:49:04.419046 11433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:04.420572 11445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:04.427009 11445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:04.437343 11445 catalog_manager.cc:1347] Generated new cluster ID: a5945081dc654bffa3abb607cef26281
I20260808 11:49:04.437477 11445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:04.449980 11445 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:04.451274 11445 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:04.457451 11445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c7bc49d405d4967aac6cf3ea2855574: Generated new TSK 0
I20260808 11:49:04.458196 11445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:04.468387 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11312
W20260808 11:49:04.470136 11404 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44981
W20260808 11:49:04.470918 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11312: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:04.471115 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 11312
W20260808 11:49:04.496847 11288 connection.cc:535] client connection to 127.0.0.1:36331 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36331): Connection reset by peer (error 104)
2026-08-08T11:49:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:04 dist-test-slave-39l8 krb5kdc[11461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:04 dist-test-slave-39l8 krb5kdc[11461](info): set up 2 sockets
Aug 08 11:49:04 dist-test-slave-39l8 krb5kdc[11461](info): commencing operation
krb5kdc: starting...
W20260808 11:49:05.762897 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:05 dist-test-slave-39l8 krb5kdc[11461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:06.066054 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42559
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:06.401607 11484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:06.401892 11484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:06.401985 11484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:06.409137 11484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:06.409243 11484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:06.409288 11484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:06.409368 11484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:06.409446 11484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:06.419493 11484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:06.420783 11484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:06.422070 11484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:06.434026 11492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:06.434000 11495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:06.434360 11491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:06.434831 11484 server_base.cc:1034] running on GCE node
I20260808 11:49:06.435618 11484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:06.436970 11484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:06.438212 11484 hybrid_clock.cc:648] HybridClock initialized: now 1786189746438173 us; error 51 us; skew 500 ppm
I20260808 11:49:06.438553 11484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:06 dist-test-slave-39l8 krb5kdc[11461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189746, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:06.441596 11484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:06.443041 11484 webserver.cc:460] Webserver started at http://127.0.0.1:45931/ using document root <none> and password file <none>
I20260808 11:49:06.443691 11484 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:06.443835 11484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:06.444155 11484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:06.447083 11484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "9d0877225749449dafb4d9eac3852447"
format_stamp: "Formatted at 2026-08-08 11:49:06 on dist-test-slave-39l8"
server_key: "43971328b827c3114eb2deeee892164f"
server_key_iv: "14d8975878a8e4ebbb96a16260438f1b"
server_key_version: "encryptionkey@0"
I20260808 11:49:06.447844 11484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "9d0877225749449dafb4d9eac3852447"
format_stamp: "Formatted at 2026-08-08 11:49:06 on dist-test-slave-39l8"
server_key: "43971328b827c3114eb2deeee892164f"
server_key_iv: "14d8975878a8e4ebbb96a16260438f1b"
server_key_version: "encryptionkey@0"
I20260808 11:49:06.452821 11484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:49:06.456147 11506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:06.457461 11484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:49:06.457641 11484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "9d0877225749449dafb4d9eac3852447"
format_stamp: "Formatted at 2026-08-08 11:49:06 on dist-test-slave-39l8"
server_key: "43971328b827c3114eb2deeee892164f"
server_key_iv: "14d8975878a8e4ebbb96a16260438f1b"
server_key_version: "encryptionkey@0"
I20260808 11:49:06.457844 11484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:06.463418 11484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:06.467132 11484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:06.467424 11484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:06.485220 11484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42559
I20260808 11:49:06.485265 11593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42559 every 8 connection(s)
I20260808 11:49:06.486615 11484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:06.487794 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 11484
I20260808 11:49:06.487995 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:06.488454 13945 external_mini_cluster.cc:1428] Setting key 69bd3902920de93b6498f4c4c2b83c65
I20260808 11:49:06.490669 11595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:06.498111 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:06.490460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49299 (local address 127.0.0.1:42559)
0808 11:49:06.492127 (+ 1667us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:06.492142 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:06.492193 (+ 51us) server_negotiation.cc:407] Connection header received
0808 11:49:06.494126 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:06.494152 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:06.494576 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:06.495045 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:06.496035 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:06.496043 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:06.496060 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:06.496087 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:06.496185 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:06.496224 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:06.496438 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:06.496587 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 11:49:06.496795 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1378,"thread_start_us":1118,"threads_started":1}
I20260808 11:49:06.501782 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:06.505348 11595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447: Bootstrap starting.
I20260808 11:49:06.509096 11595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:06.509836 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:06.507157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59323 (local address 127.0.0.1:42559)
0808 11:49:06.507324 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:06.507330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:06.507762 (+ 432us) server_negotiation.cc:407] Connection header received
0808 11:49:06.507892 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:06.507896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:06.507960 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:06.508066 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:06.509025 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:06.509030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:06.509036 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:06.509077 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:06.509126 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:06.509147 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:06.509266 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:06.509413 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 11:49:06.509470 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 11:49:06.510504 11595 log.cc:824] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:06.513175 11595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447: No bootstrap required, opened a new log
I20260808 11:49:06.520370 11595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0877225749449dafb4d9eac3852447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42559 } }
I20260808 11:49:06.520721 11595 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:06.520798 11595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0877225749449dafb4d9eac3852447, State: Initialized, Role: FOLLOWER
I20260808 11:49:06.521248 11595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0877225749449dafb4d9eac3852447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42559 } }
I20260808 11:49:06.521435 11595 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:06.521528 11595 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:06.521656 11595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:06.524163 11595 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0877225749449dafb4d9eac3852447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42559 } }
I20260808 11:49:06.524572 11595 leader_election.cc:302] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d0877225749449dafb4d9eac3852447; no voters:
I20260808 11:49:06.525061 11595 leader_election.cc:288] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:06.525204 11610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:06.525893 11610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [term 1 LEADER]: Becoming Leader. State: Replica: 9d0877225749449dafb4d9eac3852447, State: Running, Role: LEADER
I20260808 11:49:06.526358 11610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0877225749449dafb4d9eac3852447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42559 } }
I20260808 11:49:06.527089 11595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:06.529248 11614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d0877225749449dafb4d9eac3852447. Latest consensus state: current_term: 1 leader_uuid: "9d0877225749449dafb4d9eac3852447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0877225749449dafb4d9eac3852447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42559 } } }
I20260808 11:49:06.529719 11614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:06.531317 11622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:06.531255 11612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d0877225749449dafb4d9eac3852447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0877225749449dafb4d9eac3852447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42559 } } }
I20260808 11:49:06.531587 11612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:06.536535 11622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:06.545210 11622 catalog_manager.cc:1347] Generated new cluster ID: 41f813b6d9a34e96bee8ff747ac99271
I20260808 11:49:06.545336 11622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:06.568116 11622 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:06.568569 11622 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:06.581116 11622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d0877225749449dafb4d9eac3852447: Generated new TSK 0
I20260808 11:49:06.581884 11622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:06.595453 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11484
W20260808 11:49:06.596585 11588 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49299
W20260808 11:49:06.597338 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11484: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:06.597487 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 11484
2026-08-08T11:49:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2129 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:06 dist-test-slave-39l8 krb5kdc[11643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:06 dist-test-slave-39l8 krb5kdc[11643](info): set up 2 sockets
Aug 08 11:49:06 dist-test-slave-39l8 krb5kdc[11643](info): commencing operation
krb5kdc: starting...
W20260808 11:49:07.882967 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:08 dist-test-slave-39l8 krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:08.182310 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:08.512728 11658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:08.513048 11658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:08.513234 11658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:08.519917 11658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:08.520058 11658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:08.520165 11658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:08.520310 11658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:08.520390 11658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:08.529294 11658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:08.530442 11658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:08.531674 11658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:08.543349 11663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:08.543401 11668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:08.543429 11664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:08.544518 11658 server_base.cc:1034] running on GCE node
I20260808 11:49:08.545171 11658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:08.546368 11658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:08.547566 11658 hybrid_clock.cc:648] HybridClock initialized: now 1786189748547511 us; error 48 us; skew 500 ppm
I20260808 11:49:08.547854 11658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:08 dist-test-slave-39l8 krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189748, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:08.550863 11658 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:08.552248 11658 webserver.cc:460] Webserver started at http://127.0.0.1:36907/ using document root <none> and password file <none>
I20260808 11:49:08.552843 11658 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:08.552944 11658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:08.553172 11658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:08.555891 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "9c50e9e08b324ed8beeb76c2ad35fce9"
format_stamp: "Formatted at 2026-08-08 11:49:08 on dist-test-slave-39l8"
server_key: "7cba5b25eed1944384d6a50ea875b098"
server_key_iv: "857e086c32cde3cb447afae1152bd03d"
server_key_version: "encryptionkey@0"
I20260808 11:49:08.556552 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "9c50e9e08b324ed8beeb76c2ad35fce9"
format_stamp: "Formatted at 2026-08-08 11:49:08 on dist-test-slave-39l8"
server_key: "7cba5b25eed1944384d6a50ea875b098"
server_key_iv: "857e086c32cde3cb447afae1152bd03d"
server_key_version: "encryptionkey@0"
I20260808 11:49:08.561283 11658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 11:49:08.564424 11682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:08.565618 11658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:49:08.565739 11658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "9c50e9e08b324ed8beeb76c2ad35fce9"
format_stamp: "Formatted at 2026-08-08 11:49:08 on dist-test-slave-39l8"
server_key: "7cba5b25eed1944384d6a50ea875b098"
server_key_iv: "857e086c32cde3cb447afae1152bd03d"
server_key_version: "encryptionkey@0"
I20260808 11:49:08.565896 11658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:08.589187 11658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:08.593330 11658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:08.593544 11658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:08.612179 11658 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46641
I20260808 11:49:08.612231 11752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46641 every 8 connection(s)
I20260808 11:49:08.613518 11658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:08.617126 11754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:08.618294 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 11658
I20260808 11:49:08.618494 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:08.619103 13945 external_mini_cluster.cc:1428] Setting key 5690710fc4fbbe69aefc8f24825f9ab2
I20260808 11:49:08.629406 11754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9: Bootstrap starting.
Aug 08 11:49:08 dist-test-slave-39l8 krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 11:49:08.632359 11754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:08.633349 11754 log.cc:824] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:08.636394 11754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9: No bootstrap required, opened a new log
I20260808 11:49:08.640616 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:08.620979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55931 (local address 127.0.0.1:46641)
0808 11:49:08.621741 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:08.621752 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:08.621797 (+ 45us) server_negotiation.cc:407] Connection header received
0808 11:49:08.623529 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:08.623549 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:08.623885 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:08.624281 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:08.625724 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:08.627109 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:08.628367 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:08.628998 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:08.629221 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:08.633304 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:08.633311 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:08.633323 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:08.633363 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:08.636665 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:08.637345 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:08.637357 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:08.637367 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:08.637548 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:08.637974 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:08.637979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:08.637982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:08.638553 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:08.638878 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:08.639333 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 11:49:08.639710 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":343,"threads_started":1}
I20260808 11:49:08.644325 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:08.644654 11754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c50e9e08b324ed8beeb76c2ad35fce9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } }
I20260808 11:49:08.645004 11754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:08.645119 11754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c50e9e08b324ed8beeb76c2ad35fce9, State: Initialized, Role: FOLLOWER
I20260808 11:49:08.645713 11754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c50e9e08b324ed8beeb76c2ad35fce9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } }
I20260808 11:49:08.645959 11754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:08.646103 11754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:08.646277 11754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:08.648627 11754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c50e9e08b324ed8beeb76c2ad35fce9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } }
I20260808 11:49:08.649231 11754 leader_election.cc:302] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c50e9e08b324ed8beeb76c2ad35fce9; no voters:
I20260808 11:49:08.650031 11754 leader_election.cc:288] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:08.650120 11769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:08.650838 11769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [term 1 LEADER]: Becoming Leader. State: Replica: 9c50e9e08b324ed8beeb76c2ad35fce9, State: Running, Role: LEADER
I20260808 11:49:08.651456 11769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c50e9e08b324ed8beeb76c2ad35fce9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } }
I20260808 11:49:08.653426 11769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c50e9e08b324ed8beeb76c2ad35fce9. Latest consensus state: current_term: 1 leader_uuid: "9c50e9e08b324ed8beeb76c2ad35fce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c50e9e08b324ed8beeb76c2ad35fce9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } } }
I20260808 11:49:08.653757 11769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:08.654266 11771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c50e9e08b324ed8beeb76c2ad35fce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c50e9e08b324ed8beeb76c2ad35fce9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } } }
I20260808 11:49:08.654630 11771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:08.658514 11754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:08.666791 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:08.649160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35077 (local address 127.0.0.1:46641)
0808 11:49:08.649347 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:08.649351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:08.649884 (+ 533us) server_negotiation.cc:407] Connection header received
0808 11:49:08.650073 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:08.650079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:08.650162 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:08.650277 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:08.652365 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:08.653876 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:08.656365 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:08.657069 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:08.657195 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:08.660922 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:08.660926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:08.660929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:08.660957 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:08.663715 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:08.664364 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:08.664369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:08.664379 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:08.664507 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:08.664897 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:08.664901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:08.664903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:08.665223 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:08.665476 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:08.665809 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 11:49:08.666010 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:49:08.666960 11774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:08.672457 11774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:08.682468 11774 catalog_manager.cc:1347] Generated new cluster ID: 8f128b15b76e4711b0e7a705d96e65b9
I20260808 11:49:08.682586 11774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:08.703675 11774 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:08.705072 11774 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:08.713191 11774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c50e9e08b324ed8beeb76c2ad35fce9: Generated new TSK 0
I20260808 11:49:08.713989 11774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:08.720314 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11658
I20260808 11:49:08.721469 11748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:08.810721 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 11658
2026-08-08T11:49:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:08 dist-test-slave-39l8 krb5kdc[11798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:08 dist-test-slave-39l8 krb5kdc[11798](info): set up 2 sockets
Aug 08 11:49:08 dist-test-slave-39l8 krb5kdc[11798](info): commencing operation
krb5kdc: starting...
W20260808 11:49:10.111718 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:10 dist-test-slave-39l8 krb5kdc[11798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:10.411885 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:10.746129 11817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:10.746404 11817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:10.746567 11817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:10.753770 11817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:10.753873 11817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:10.753983 11817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:10.754083 11817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:10.754159 11817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:10.764305 11817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:10.765609 11817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:10.767028 11817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:10.778875 11825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:10.779022 11833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:10.779081 11827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:10.792412 11817 server_base.cc:1034] running on GCE node
I20260808 11:49:10.793325 11817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:10.794916 11817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:10.796147 11817 hybrid_clock.cc:648] HybridClock initialized: now 1786189750796107 us; error 50 us; skew 500 ppm
I20260808 11:49:10.796494 11817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:10 dist-test-slave-39l8 krb5kdc[11798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189750, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:10.802531 11817 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:10.804177 11817 webserver.cc:460] Webserver started at http://127.0.0.1:34369/ using document root <none> and password file <none>
I20260808 11:49:10.804896 11817 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:10.805044 11817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:10.805354 11817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:10.808519 11817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "7ea8972dc4104acc96738485d7ecd290"
format_stamp: "Formatted at 2026-08-08 11:49:10 on dist-test-slave-39l8"
server_key: "e26d81f458d97ec0b5dcbd0bfe817d60"
server_key_iv: "22cc2fe879356351a0fc8ea830b9756f"
server_key_version: "encryptionkey@0"
I20260808 11:49:10.809290 11817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "7ea8972dc4104acc96738485d7ecd290"
format_stamp: "Formatted at 2026-08-08 11:49:10 on dist-test-slave-39l8"
server_key: "e26d81f458d97ec0b5dcbd0bfe817d60"
server_key_iv: "22cc2fe879356351a0fc8ea830b9756f"
server_key_version: "encryptionkey@0"
I20260808 11:49:10.814358 11817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:49:10.817836 11846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:10.819136 11817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:10.819319 11817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "7ea8972dc4104acc96738485d7ecd290"
format_stamp: "Formatted at 2026-08-08 11:49:10 on dist-test-slave-39l8"
server_key: "e26d81f458d97ec0b5dcbd0bfe817d60"
server_key_iv: "22cc2fe879356351a0fc8ea830b9756f"
server_key_version: "encryptionkey@0"
I20260808 11:49:10.819521 11817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:10.837894 11817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:10.842110 11817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:10.842378 11817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:10.860567 11817 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35151
I20260808 11:49:10.860596 11923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35151 every 8 connection(s)
I20260808 11:49:10.861815 11817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:10.865290 11925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:10.872117 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 11817
I20260808 11:49:10.872323 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:10.872905 13945 external_mini_cluster.cc:1428] Setting key c847abde72f354ea9ff69721d4ab574a
I20260808 11:49:10.875949 11925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290: Bootstrap starting.
I20260808 11:49:10.879340 11925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:10.880445 11925 log.cc:824] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:10.883342 11925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290: No bootstrap required, opened a new log
I20260808 11:49:10.886183 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:10.874729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37761 (local address 127.0.0.1:35151)
0808 11:49:10.875546 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:10.875557 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:10.875599 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:49:10.877401 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:10.877424 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:10.877792 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:10.878101 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:10.878828 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:10.880219 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:10.881824 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:10.882465 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:10.882590 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:10.883907 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:10.883915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:10.883928 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:10.884435 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:10.884508 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:10.884532 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:10.884622 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:10.884860 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 11:49:10.885175 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":398,"threads_started":1}
I20260808 11:49:10.889889 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:10.892125 11925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea8972dc4104acc96738485d7ecd290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35151 } }
I20260808 11:49:10.892575 11925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:10.892690 11925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ea8972dc4104acc96738485d7ecd290, State: Initialized, Role: FOLLOWER
I20260808 11:49:10.893359 11925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea8972dc4104acc96738485d7ecd290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35151 } }
I20260808 11:49:10.893662 11925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:10.893855 11925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:10.894073 11925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:10.896663 11925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea8972dc4104acc96738485d7ecd290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35151 } }
I20260808 11:49:10.897524 11925 leader_election.cc:302] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ea8972dc4104acc96738485d7ecd290; no voters:
I20260808 11:49:10.898375 11925 leader_election.cc:288] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:10.898411 11935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:10.899843 11935 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [term 1 LEADER]: Becoming Leader. State: Replica: 7ea8972dc4104acc96738485d7ecd290, State: Running, Role: LEADER
I20260808 11:49:10.900490 11935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea8972dc4104acc96738485d7ecd290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35151 } }
I20260808 11:49:10.900847 11925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:10.903683 11937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ea8972dc4104acc96738485d7ecd290. Latest consensus state: current_term: 1 leader_uuid: "7ea8972dc4104acc96738485d7ecd290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea8972dc4104acc96738485d7ecd290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35151 } } }
I20260808 11:49:10.904172 11937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:10.906242 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:10.895343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51147 (local address 127.0.0.1:35151)
0808 11:49:10.895533 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:10.895541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:10.897622 (+ 2081us) server_negotiation.cc:407] Connection header received
0808 11:49:10.897815 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:10.897818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:10.897880 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:10.897953 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:10.898775 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:10.900186 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:10.902256 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:10.903329 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:10.903462 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:10.904894 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:10.904900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:10.904904 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:10.904924 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:10.904966 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:10.904986 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:10.905099 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:10.905633 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 11:49:10.905770 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:49:10.909276 11938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ea8972dc4104acc96738485d7ecd290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea8972dc4104acc96738485d7ecd290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35151 } } }
I20260808 11:49:10.909659 11938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:10.911880 11945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:10.917106 11945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:10.926111 11945 catalog_manager.cc:1347] Generated new cluster ID: 88f609a4f3444825bf89cc137eb021e1
I20260808 11:49:10.926226 11945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:10.946306 11945 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:10.947719 11945 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:10.957571 11945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ea8972dc4104acc96738485d7ecd290: Generated new TSK 0
I20260808 11:49:10.958546 11945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:10.973623 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11817
W20260808 11:49:10.974658 11919 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37761
W20260808 11:49:10.975515 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11817: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:10.975675 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 11817
2026-08-08T11:49:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2164 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:11 dist-test-slave-39l8 krb5kdc[11967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:11 dist-test-slave-39l8 krb5kdc[11967](info): set up 2 sockets
Aug 08 11:49:11 dist-test-slave-39l8 krb5kdc[11967](info): commencing operation
krb5kdc: starting...
W20260808 11:49:12.260589 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:12 dist-test-slave-39l8 krb5kdc[11967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:12.568307 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:12.906423 11986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:12.906773 11986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:12.906965 11986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:12.915230 11986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:12.915375 11986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:12.915481 11986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:12.915599 11986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:12.915684 11986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:12.925349 11986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:12.926784 11986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:12.928119 11986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:12.940323 12000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:12.940425 11997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:12.940951 11996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:12.941052 11986 server_base.cc:1034] running on GCE node
I20260808 11:49:12.941875 11986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:12.943291 11986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:12.944530 11986 hybrid_clock.cc:648] HybridClock initialized: now 1786189752944498 us; error 45 us; skew 500 ppm
I20260808 11:49:12.944895 11986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:12 dist-test-slave-39l8 krb5kdc[11967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:12.948068 11986 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:12.949559 11986 webserver.cc:460] Webserver started at http://127.0.0.1:46657/ using document root <none> and password file <none>
I20260808 11:49:12.950235 11986 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:12.950378 11986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:12.950686 11986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:12.953647 11986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "1532bb2e427e41d6973ec54d82e1d96a"
format_stamp: "Formatted at 2026-08-08 11:49:12 on dist-test-slave-39l8"
server_key: "03e2e61537aba79ce2233aafb1a8b8e5"
server_key_iv: "43da7b2f1b97a6b5c96efec10e337089"
server_key_version: "encryptionkey@0"
I20260808 11:49:12.954384 11986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "1532bb2e427e41d6973ec54d82e1d96a"
format_stamp: "Formatted at 2026-08-08 11:49:12 on dist-test-slave-39l8"
server_key: "03e2e61537aba79ce2233aafb1a8b8e5"
server_key_iv: "43da7b2f1b97a6b5c96efec10e337089"
server_key_version: "encryptionkey@0"
I20260808 11:49:12.959374 11986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 11:49:12.962540 12010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:12.963812 11986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:49:12.963974 11986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "1532bb2e427e41d6973ec54d82e1d96a"
format_stamp: "Formatted at 2026-08-08 11:49:12 on dist-test-slave-39l8"
server_key: "03e2e61537aba79ce2233aafb1a8b8e5"
server_key_iv: "43da7b2f1b97a6b5c96efec10e337089"
server_key_version: "encryptionkey@0"
I20260808 11:49:12.964149 11986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:12.969544 11986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:12.973250 11986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:12.973521 11986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:12.991619 11986 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41549
I20260808 11:49:12.991698 12098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41549 every 8 connection(s)
I20260808 11:49:12.993002 11986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:12.996629 12099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:13.000556 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 11986
I20260808 11:49:13.000751 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:13.001164 13945 external_mini_cluster.cc:1428] Setting key 29c8cc3f1d818db6c80910859b8292cf
I20260808 11:49:13.009210 12106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:13.002902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41293 (local address 127.0.0.1:41549)
0808 11:49:13.003706 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:13.003721 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:13.003772 (+ 51us) server_negotiation.cc:407] Connection header received
0808 11:49:13.005453 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:13.005486 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:13.005917 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:13.006347 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:13.007345 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:13.007354 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:13.007371 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:13.007388 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:13.007460 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:13.007484 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:13.007698 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:13.007830 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 11:49:13.008022 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":327,"threads_started":1}
I20260808 11:49:13.010015 12099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a: Bootstrap starting.
I20260808 11:49:13.012743 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:13.013360 12099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:13.014631 12099 log.cc:824] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:13.017956 12099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a: No bootstrap required, opened a new log
I20260808 11:49:13.020423 12106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:13.017936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52767 (local address 127.0.0.1:41549)
0808 11:49:13.018102 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:13.018108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:13.018450 (+ 342us) server_negotiation.cc:407] Connection header received
0808 11:49:13.018556 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:13.018567 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:13.018626 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:13.018765 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:13.019694 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:13.019700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:13.019704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:13.019722 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:13.019773 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:13.019799 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:13.019938 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:13.020054 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 11:49:13.020114 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:49:13.026072 12099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1532bb2e427e41d6973ec54d82e1d96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } }
I20260808 11:49:13.026433 12099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:13.026587 12099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1532bb2e427e41d6973ec54d82e1d96a, State: Initialized, Role: FOLLOWER
I20260808 11:49:13.027305 12099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1532bb2e427e41d6973ec54d82e1d96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } }
I20260808 11:49:13.027525 12099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:13.027622 12099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:13.027746 12099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:13.029780 12099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1532bb2e427e41d6973ec54d82e1d96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } }
I20260808 11:49:13.030211 12099 leader_election.cc:302] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1532bb2e427e41d6973ec54d82e1d96a; no voters:
I20260808 11:49:13.030881 12099 leader_election.cc:288] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:13.030992 12116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:13.032461 12116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [term 1 LEADER]: Becoming Leader. State: Replica: 1532bb2e427e41d6973ec54d82e1d96a, State: Running, Role: LEADER
I20260808 11:49:13.033068 12099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:13.032946 12116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1532bb2e427e41d6973ec54d82e1d96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } }
I20260808 11:49:13.036134 12118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1532bb2e427e41d6973ec54d82e1d96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1532bb2e427e41d6973ec54d82e1d96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } } }
I20260808 11:49:13.036728 12118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:13.036130 12119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1532bb2e427e41d6973ec54d82e1d96a. Latest consensus state: current_term: 1 leader_uuid: "1532bb2e427e41d6973ec54d82e1d96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1532bb2e427e41d6973ec54d82e1d96a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } } }
I20260808 11:49:13.037055 12119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:13.039238 12130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:13.045433 12130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:13.053918 12130 catalog_manager.cc:1347] Generated new cluster ID: 4fd1e7b34f134b82b93d9673c25802ac
I20260808 11:49:13.054044 12130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:13.072615 12130 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:13.073071 12130 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:13.081063 12130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1532bb2e427e41d6973ec54d82e1d96a: Generated new TSK 0
I20260808 11:49:13.081828 12130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:13.091223 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11986
W20260808 11:49:13.092372 12091 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41293
W20260808 11:49:13.093080 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 11986: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:13.093250 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 11986
2026-08-08T11:49:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2115 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:13 dist-test-slave-39l8 krb5kdc[12147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:13 dist-test-slave-39l8 krb5kdc[12147](info): set up 2 sockets
Aug 08 11:49:13 dist-test-slave-39l8 krb5kdc[12147](info): commencing operation
krb5kdc: starting...
W20260808 11:49:14.381804 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:14 dist-test-slave-39l8 krb5kdc[12147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:14.687587 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:15.020937 12166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:15.021219 12166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:15.021333 12166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:15.028183 12166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:15.028311 12166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:15.028422 12166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:15.028538 12166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:15.028626 12166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:15.038048 12166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:15.039348 12166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:15.040624 12166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:15.052243 12174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:15.052381 12173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:15.052402 12176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:15.053377 12166 server_base.cc:1034] running on GCE node
I20260808 11:49:15.054140 12166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:15.055465 12166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:15.056659 12166 hybrid_clock.cc:648] HybridClock initialized: now 1786189755056630 us; error 28 us; skew 500 ppm
I20260808 11:49:15.056986 12166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:15 dist-test-slave-39l8 krb5kdc[12147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189755, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:15.060068 12166 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:15.061462 12166 webserver.cc:460] Webserver started at http://127.0.0.1:41743/ using document root <none> and password file <none>
I20260808 11:49:15.062166 12166 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:15.062318 12166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:15.062639 12166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:15.065420 12166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "1e779ce678b04650871c5e1bca3d44d9"
format_stamp: "Formatted at 2026-08-08 11:49:15 on dist-test-slave-39l8"
server_key: "48e5598fd8c39d2b07eec35241643ee5"
server_key_iv: "dbcffb76d43083f338142fd8b2dd3506"
server_key_version: "encryptionkey@0"
I20260808 11:49:15.066176 12166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "1e779ce678b04650871c5e1bca3d44d9"
format_stamp: "Formatted at 2026-08-08 11:49:15 on dist-test-slave-39l8"
server_key: "48e5598fd8c39d2b07eec35241643ee5"
server_key_iv: "dbcffb76d43083f338142fd8b2dd3506"
server_key_version: "encryptionkey@0"
I20260808 11:49:15.071111 12166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 11:49:15.074451 12187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:15.075745 12166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:15.075925 12166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "1e779ce678b04650871c5e1bca3d44d9"
format_stamp: "Formatted at 2026-08-08 11:49:15 on dist-test-slave-39l8"
server_key: "48e5598fd8c39d2b07eec35241643ee5"
server_key_iv: "dbcffb76d43083f338142fd8b2dd3506"
server_key_version: "encryptionkey@0"
I20260808 11:49:15.076108 12166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:15.089082 12166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:15.092675 12166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:15.092965 12166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:15.111148 12166 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33889
I20260808 11:49:15.111187 12270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33889 every 8 connection(s)
I20260808 11:49:15.112560 12166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:15.116254 12271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:15.119059 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 12166
I20260808 11:49:15.119266 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:15.119846 13945 external_mini_cluster.cc:1428] Setting key 62cf73a5f2e9b7012dc4e9786b4e14cf
I20260808 11:49:15.127893 12271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9: Bootstrap starting.
I20260808 11:49:15.130937 12271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:15.131875 12271 log.cc:824] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:49:15 dist-test-slave-39l8 krb5kdc[12147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 11:49:15.135140 12271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9: No bootstrap required, opened a new log
I20260808 11:49:15.142434 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:15.121914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:43123 (local address 127.0.0.1:33889)
0808 11:49:15.122663 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:15.122675 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:15.122766 (+ 91us) server_negotiation.cc:407] Connection header received
0808 11:49:15.124398 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:15.124418 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:15.124796 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:15.125190 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:15.126620 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:15.127926 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:15.129277 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:15.129930 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:15.130120 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:15.134992 (+ 4872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:15.134999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:15.135012 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:15.135049 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:15.138065 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:15.139084 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:15.139089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:15.139095 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:15.139272 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:15.139806 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:15.139809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:15.139811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:15.140318 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:15.140550 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:15.141060 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 11:49:15.141369 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":336,"threads_started":1}
I20260808 11:49:15.144299 12271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e779ce678b04650871c5e1bca3d44d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260808 11:49:15.144732 12271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:15.144901 12271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e779ce678b04650871c5e1bca3d44d9, State: Initialized, Role: FOLLOWER
I20260808 11:49:15.145416 12271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e779ce678b04650871c5e1bca3d44d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260808 11:49:15.145694 12271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:15.145846 12271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:15.146029 12271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:15.146534 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:15.148329 12271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e779ce678b04650871c5e1bca3d44d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260808 11:49:15.148797 12271 leader_election.cc:302] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e779ce678b04650871c5e1bca3d44d9; no voters:
I20260808 11:49:15.149431 12271 leader_election.cc:288] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:15.149539 12278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:15.151175 12278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [term 1 LEADER]: Becoming Leader. State: Replica: 1e779ce678b04650871c5e1bca3d44d9, State: Running, Role: LEADER
I20260808 11:49:15.151855 12271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:15.151783 12278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e779ce678b04650871c5e1bca3d44d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260808 11:49:15.156971 12284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e779ce678b04650871c5e1bca3d44d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e779ce678b04650871c5e1bca3d44d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } } }
I20260808 11:49:15.157423 12284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:15.158016 12291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e779ce678b04650871c5e1bca3d44d9. Latest consensus state: current_term: 1 leader_uuid: "1e779ce678b04650871c5e1bca3d44d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e779ce678b04650871c5e1bca3d44d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } } }
I20260808 11:49:15.158389 12291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:15.169983 12297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:15.173496 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:15.152074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:32911 (local address 127.0.0.1:33889)
0808 11:49:15.152299 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:15.152308 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:15.155259 (+ 2951us) server_negotiation.cc:407] Connection header received
0808 11:49:15.155494 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:15.155502 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:15.155593 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:15.155694 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:15.157993 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:15.159555 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:15.161903 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:15.162808 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:15.162932 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:15.165925 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:15.165930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:15.165933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:15.165961 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:15.168903 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:15.169824 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:15.169829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:15.169832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:15.169978 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:15.170521 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:15.170528 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:15.170531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:15.170973 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:15.171303 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:15.172695 (+ 1392us) server_negotiation.cc:299] Negotiation successful
0808 11:49:15.172937 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:49:15.175900 12297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:15.184918 12297 catalog_manager.cc:1347] Generated new cluster ID: 95a449c647a148ecac3bcc11ff9f076f
I20260808 11:49:15.185035 12297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:15.198293 12297 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:15.199572 12297 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:15.210764 12297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e779ce678b04650871c5e1bca3d44d9: Generated new TSK 0
I20260808 11:49:15.211632 12297 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:15.216976 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 12166
I20260808 11:49:15.218148 12264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:15.306449 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 12166
W20260808 11:49:15.329979 12145 connection.cc:535] client connection to 127.0.0.1:33889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33889): Connection reset by peer (error 104)
2026-08-08T11:49:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2213 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:15 dist-test-slave-39l8 krb5kdc[12324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:15 dist-test-slave-39l8 krb5kdc[12324](info): set up 2 sockets
Aug 08 11:49:15 dist-test-slave-39l8 krb5kdc[12324](info): commencing operation
krb5kdc: starting...
W20260808 11:49:16.604140 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:16 dist-test-slave-39l8 krb5kdc[12324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:16.908218 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:17.240832 12344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:17.241096 12344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:17.241217 12344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:17.248149 12344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:17.248287 12344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:17.248389 12344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:17.248504 12344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:17.248587 12344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:17.258395 12344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:17.259719 12344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:17.261011 12344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:17.272603 12354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:17.272567 12356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:17.272647 12352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:17.273351 12344 server_base.cc:1034] running on GCE node
I20260808 11:49:17.274104 12344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:17.275390 12344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:17.276575 12344 hybrid_clock.cc:648] HybridClock initialized: now 1786189757276520 us; error 43 us; skew 500 ppm
I20260808 11:49:17.276888 12344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:17 dist-test-slave-39l8 krb5kdc[12324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189757, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:17.283531 12344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:17.284870 12344 webserver.cc:460] Webserver started at http://127.0.0.1:43267/ using document root <none> and password file <none>
I20260808 11:49:17.285519 12344 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:17.285666 12344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:17.285969 12344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:17.288980 12344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "27efbc8abc5d4a73b1cedd23160ac820"
format_stamp: "Formatted at 2026-08-08 11:49:17 on dist-test-slave-39l8"
server_key: "52302adf23217742b36b0fb1ca74ecb1"
server_key_iv: "f6f9e228637bc1f9c4beb22af65593ae"
server_key_version: "encryptionkey@0"
I20260808 11:49:17.289762 12344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "27efbc8abc5d4a73b1cedd23160ac820"
format_stamp: "Formatted at 2026-08-08 11:49:17 on dist-test-slave-39l8"
server_key: "52302adf23217742b36b0fb1ca74ecb1"
server_key_iv: "f6f9e228637bc1f9c4beb22af65593ae"
server_key_version: "encryptionkey@0"
I20260808 11:49:17.294597 12344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260808 11:49:17.298175 12364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:17.299400 12344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:17.299571 12344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "27efbc8abc5d4a73b1cedd23160ac820"
format_stamp: "Formatted at 2026-08-08 11:49:17 on dist-test-slave-39l8"
server_key: "52302adf23217742b36b0fb1ca74ecb1"
server_key_iv: "f6f9e228637bc1f9c4beb22af65593ae"
server_key_version: "encryptionkey@0"
I20260808 11:49:17.299748 12344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:17.313647 12344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:17.317389 12344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:17.317677 12344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:17.335765 12344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46107
I20260808 11:49:17.335804 12437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46107 every 8 connection(s)
I20260808 11:49:17.337023 12344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:17.337594 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 12344
I20260808 11:49:17.337800 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:17.338450 13945 external_mini_cluster.cc:1428] Setting key 781a00f5090b5d689941259be05ec69b
I20260808 11:49:17.341511 12439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:17.351289 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:17.340352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:37779 (local address 127.0.0.1:46107)
0808 11:49:17.341061 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:17.341072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:17.341110 (+ 38us) server_negotiation.cc:407] Connection header received
0808 11:49:17.342826 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:17.342848 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:17.343233 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:17.343582 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:17.344154 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:17.345694 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:17.346951 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:17.347583 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:17.347793 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:17.349085 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:17.349092 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:17.349104 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:17.349593 (+ 489us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:17.349671 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:17.349696 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:17.349877 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:17.350156 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 11:49:17.350335 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":308,"threads_started":1}
I20260808 11:49:17.352397 12439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820: Bootstrap starting.
I20260808 11:49:17.355185 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:17.355950 12439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:17.356907 12439 log.cc:824] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:17.359923 12439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820: No bootstrap required, opened a new log
I20260808 11:49:17.365794 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:17.359603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:34443 (local address 127.0.0.1:46107)
0808 11:49:17.359775 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:17.359780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:17.360212 (+ 432us) server_negotiation.cc:407] Connection header received
0808 11:49:17.360343 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:17.360348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:17.360431 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:17.360538 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:17.360951 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:17.362032 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:17.363334 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:17.363952 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:17.364058 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:17.365052 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:17.365058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:17.365062 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:17.365081 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:17.365124 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:17.365142 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:17.365278 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:17.365426 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 11:49:17.365476 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:49:17.368875 12439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27efbc8abc5d4a73b1cedd23160ac820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46107 } }
I20260808 11:49:17.369326 12439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:17.369465 12439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27efbc8abc5d4a73b1cedd23160ac820, State: Initialized, Role: FOLLOWER
I20260808 11:49:17.370044 12439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27efbc8abc5d4a73b1cedd23160ac820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46107 } }
I20260808 11:49:17.370283 12439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:17.370440 12439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:17.370625 12439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:17.372776 12439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27efbc8abc5d4a73b1cedd23160ac820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46107 } }
I20260808 11:49:17.373270 12439 leader_election.cc:302] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27efbc8abc5d4a73b1cedd23160ac820; no voters:
I20260808 11:49:17.373828 12439 leader_election.cc:288] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:17.373942 12455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:17.374670 12455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [term 1 LEADER]: Becoming Leader. State: Replica: 27efbc8abc5d4a73b1cedd23160ac820, State: Running, Role: LEADER
I20260808 11:49:17.375371 12455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27efbc8abc5d4a73b1cedd23160ac820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46107 } }
I20260808 11:49:17.376140 12439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:17.378304 12456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27efbc8abc5d4a73b1cedd23160ac820" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27efbc8abc5d4a73b1cedd23160ac820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46107 } } }
I20260808 11:49:17.378391 12457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27efbc8abc5d4a73b1cedd23160ac820. Latest consensus state: current_term: 1 leader_uuid: "27efbc8abc5d4a73b1cedd23160ac820" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27efbc8abc5d4a73b1cedd23160ac820" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46107 } } }
I20260808 11:49:17.378731 12456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:17.378931 12457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:17.387383 12462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:17.391584 12462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:17.399092 12462 catalog_manager.cc:1347] Generated new cluster ID: 817f222743e54c3d8137c2cabebc2ab9
I20260808 11:49:17.399372 12462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:17.411803 12462 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:17.413058 12462 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:17.431178 12462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27efbc8abc5d4a73b1cedd23160ac820: Generated new TSK 0
I20260808 11:49:17.432065 12462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:17.444072 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 12344
W20260808 11:49:17.445544 12432 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:37779
W20260808 11:49:17.446456 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 12344: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:17.446646 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 12344
W20260808 11:49:17.472707 12322 connection.cc:535] client connection to 127.0.0.1:46107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46107): Connection reset by peer (error 104)
2026-08-08T11:49:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2142 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:17 dist-test-slave-39l8 krb5kdc[12486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:17 dist-test-slave-39l8 krb5kdc[12486](info): set up 2 sockets
Aug 08 11:49:17 dist-test-slave-39l8 krb5kdc[12486](info): commencing operation
krb5kdc: starting...
W20260808 11:49:18.731410 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:18 dist-test-slave-39l8 krb5kdc[12486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:19.042251 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:19.395709 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:19.396019 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:19.396167 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:19.405071 12505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:19.405170 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:19.405224 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:19.405287 12505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:19.405329 12505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:19.414819 12505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:19.415998 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:19.417272 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:19.428784 12521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:19.428813 12519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:19.428867 12523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:19.429581 12505 server_base.cc:1034] running on GCE node
I20260808 11:49:19.430366 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:19.431723 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:19.432945 12505 hybrid_clock.cc:648] HybridClock initialized: now 1786189759432905 us; error 54 us; skew 500 ppm
I20260808 11:49:19.433287 12505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:19 dist-test-slave-39l8 krb5kdc[12486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189759, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:19.436369 12505 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:19.437716 12505 webserver.cc:460] Webserver started at http://127.0.0.1:45999/ using document root <none> and password file <none>
I20260808 11:49:19.438387 12505 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:19.438534 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:19.438921 12505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:19.441735 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "b1a964c5cd7848479a94a6e220a3deca"
format_stamp: "Formatted at 2026-08-08 11:49:19 on dist-test-slave-39l8"
server_key: "faaa27b0c0ba84efe74c196057a1d65f"
server_key_iv: "4449ce11d0159ccc5a29370336114a34"
server_key_version: "encryptionkey@0"
I20260808 11:49:19.442476 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "b1a964c5cd7848479a94a6e220a3deca"
format_stamp: "Formatted at 2026-08-08 11:49:19 on dist-test-slave-39l8"
server_key: "faaa27b0c0ba84efe74c196057a1d65f"
server_key_iv: "4449ce11d0159ccc5a29370336114a34"
server_key_version: "encryptionkey@0"
I20260808 11:49:19.447365 12505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 11:49:19.450670 12532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:19.451956 12505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:49:19.452164 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "b1a964c5cd7848479a94a6e220a3deca"
format_stamp: "Formatted at 2026-08-08 11:49:19 on dist-test-slave-39l8"
server_key: "faaa27b0c0ba84efe74c196057a1d65f"
server_key_iv: "4449ce11d0159ccc5a29370336114a34"
server_key_version: "encryptionkey@0"
I20260808 11:49:19.452354 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:19.457572 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:19.461135 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:19.461431 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:19.479418 12505 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37923
I20260808 11:49:19.479493 12609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37923 every 8 connection(s)
I20260808 11:49:19.480686 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:19.483425 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 12505
I20260808 11:49:19.483610 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:19.484054 13945 external_mini_cluster.cc:1428] Setting key d0800d9aea90aec5cd66334a7d8bfc75
I20260808 11:49:19.484283 12610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:19.492652 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:19.485791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:37603 (local address 127.0.0.1:37923)
0808 11:49:19.486546 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:19.486560 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:19.486609 (+ 49us) server_negotiation.cc:407] Connection header received
0808 11:49:19.489044 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:19.489071 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:19.489471 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:19.489855 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:19.490805 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:19.490814 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:19.490830 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:19.490853 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:19.490920 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:19.490952 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:19.491153 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:19.491298 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 11:49:19.491474 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":311,"threads_started":1}
I20260808 11:49:19.496137 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:19.498286 12610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca: Bootstrap starting.
I20260808 11:49:19.501662 12610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:19.503134 12610 log.cc:824] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:19.503443 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:19.500725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:59451 (local address 127.0.0.1:37923)
0808 11:49:19.500901 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:19.500907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:19.501408 (+ 501us) server_negotiation.cc:407] Connection header received
0808 11:49:19.501600 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:19.501604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:19.501660 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:19.501758 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:19.502751 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:19.502760 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:19.502767 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:19.502794 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:19.502830 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:19.502845 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:19.502934 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:19.503085 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 11:49:19.503134 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 11:49:19.505916 12610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca: No bootstrap required, opened a new log
I20260808 11:49:19.513234 12610 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a964c5cd7848479a94a6e220a3deca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } }
I20260808 11:49:19.513583 12610 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:19.513715 12610 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a964c5cd7848479a94a6e220a3deca, State: Initialized, Role: FOLLOWER
I20260808 11:49:19.514317 12610 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a964c5cd7848479a94a6e220a3deca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } }
I20260808 11:49:19.514533 12610 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:19.514680 12610 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:19.514894 12610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:19.517385 12610 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a964c5cd7848479a94a6e220a3deca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } }
I20260808 11:49:19.517879 12610 leader_election.cc:302] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1a964c5cd7848479a94a6e220a3deca; no voters:
I20260808 11:49:19.518407 12610 leader_election.cc:288] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:19.518538 12623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:19.519241 12623 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [term 1 LEADER]: Becoming Leader. State: Replica: b1a964c5cd7848479a94a6e220a3deca, State: Running, Role: LEADER
I20260808 11:49:19.519758 12623 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a964c5cd7848479a94a6e220a3deca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } }
I20260808 11:49:19.520368 12610 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:19.522532 12628 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1a964c5cd7848479a94a6e220a3deca. Latest consensus state: current_term: 1 leader_uuid: "b1a964c5cd7848479a94a6e220a3deca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a964c5cd7848479a94a6e220a3deca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } } }
I20260808 11:49:19.522950 12628 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:19.523422 12627 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1a964c5cd7848479a94a6e220a3deca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a964c5cd7848479a94a6e220a3deca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } } }
I20260808 11:49:19.523790 12627 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:19.527690 12636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:19.533257 12636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:19.541975 12636 catalog_manager.cc:1347] Generated new cluster ID: 5adb989211f241228977c2c9e89b4f4b
I20260808 11:49:19.542081 12636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:19.556728 12636 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:19.557216 12636 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:19.568120 12636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1a964c5cd7848479a94a6e220a3deca: Generated new TSK 0
I20260808 11:49:19.568923 12636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:19.585052 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 12505
W20260808 11:49:19.586158 12601 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:37603
W20260808 11:49:19.586875 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 12505: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:19.587023 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 12505
2026-08-08T11:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:19 dist-test-slave-39l8 krb5kdc[12656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:19 dist-test-slave-39l8 krb5kdc[12656](info): set up 2 sockets
Aug 08 11:49:19 dist-test-slave-39l8 krb5kdc[12656](info): commencing operation
krb5kdc: starting...
W20260808 11:49:20.874979 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:21 dist-test-slave-39l8 krb5kdc[12656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:21.177258 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:21.507855 12672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:21.508184 12672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:21.508358 12672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:21.514994 12672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:21.515121 12672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:21.515234 12672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:21.515343 12672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:21.515431 12672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:21.524686 12672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:21.525893 12672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:21.527130 12672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:21.538841 12680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:21.538889 12681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:21.538800 12683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:21.544014 12672 server_base.cc:1034] running on GCE node
I20260808 11:49:21.544726 12672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:21.545970 12672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:21.547194 12672 hybrid_clock.cc:648] HybridClock initialized: now 1786189761547164 us; error 36 us; skew 500 ppm
I20260808 11:49:21.547539 12672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:21 dist-test-slave-39l8 krb5kdc[12656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189761, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:21.550420 12672 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:21.551941 12672 webserver.cc:460] Webserver started at http://127.0.0.1:45125/ using document root <none> and password file <none>
I20260808 11:49:21.552690 12672 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:21.552834 12672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:21.553134 12672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:21.556012 12672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "c852c178613a414289c222fd7e51b0d9"
format_stamp: "Formatted at 2026-08-08 11:49:21 on dist-test-slave-39l8"
server_key: "aff4333440c3c163952cb600d5239e56"
server_key_iv: "2312a2982ce83948120b74923c53a4db"
server_key_version: "encryptionkey@0"
I20260808 11:49:21.556763 12672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "c852c178613a414289c222fd7e51b0d9"
format_stamp: "Formatted at 2026-08-08 11:49:21 on dist-test-slave-39l8"
server_key: "aff4333440c3c163952cb600d5239e56"
server_key_iv: "2312a2982ce83948120b74923c53a4db"
server_key_version: "encryptionkey@0"
I20260808 11:49:21.561606 12672 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 11:49:21.564806 12698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:21.565972 12672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:49:21.566157 12672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "c852c178613a414289c222fd7e51b0d9"
format_stamp: "Formatted at 2026-08-08 11:49:21 on dist-test-slave-39l8"
server_key: "aff4333440c3c163952cb600d5239e56"
server_key_iv: "2312a2982ce83948120b74923c53a4db"
server_key_version: "encryptionkey@0"
I20260808 11:49:21.566342 12672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:21.600445 12672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:21.604350 12672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:21.604678 12672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:21.624126 12672 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42719
I20260808 11:49:21.624176 12774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42719 every 8 connection(s)
I20260808 11:49:21.625481 12672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:21.627921 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 12672
I20260808 11:49:21.628115 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:21.628545 13945 external_mini_cluster.cc:1428] Setting key 85de191e6ae9eb49bf069c2aff09b47c
I20260808 11:49:21.629648 12777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:49:21 dist-test-slave-39l8 krb5kdc[12656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 11:49:21.643636 12777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9: Bootstrap starting.
I20260808 11:49:21.647768 12777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:21.649224 12777 log.cc:824] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:21.652266 12777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9: No bootstrap required, opened a new log
I20260808 11:49:21.653049 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:21.630428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:51703 (local address 127.0.0.1:42719)
0808 11:49:21.631331 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:21.631348 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:21.631402 (+ 54us) server_negotiation.cc:407] Connection header received
0808 11:49:21.633498 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:21.633526 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:21.634030 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:21.634467 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:21.635801 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:21.637863 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:21.639245 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:21.640249 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:21.640445 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:21.644513 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:21.644524 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:21.644542 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:21.644584 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:21.648655 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:21.649518 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:21.649524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:21.649531 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:21.649702 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:21.650203 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:21.650206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:21.650209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:21.650756 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:21.651094 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:21.651650 (+ 556us) server_negotiation.cc:299] Negotiation successful
0808 11:49:21.651973 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":358,"threads_started":1}
I20260808 11:49:21.656965 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:21.660398 12777 raft_consensus.cc:348] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c852c178613a414289c222fd7e51b0d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42719 } }
I20260808 11:49:21.660758 12777 raft_consensus.cc:374] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:21.660880 12777 raft_consensus.cc:729] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c852c178613a414289c222fd7e51b0d9, State: Initialized, Role: FOLLOWER
I20260808 11:49:21.661602 12777 consensus_queue.cc:260] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c852c178613a414289c222fd7e51b0d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42719 } }
I20260808 11:49:21.661913 12777 raft_consensus.cc:388] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:21.662074 12777 raft_consensus.cc:482] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:21.662286 12777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:21.664608 12777 raft_consensus.cc:504] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c852c178613a414289c222fd7e51b0d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42719 } }
I20260808 11:49:21.665094 12777 leader_election.cc:302] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c852c178613a414289c222fd7e51b0d9; no voters:
I20260808 11:49:21.665723 12777 leader_election.cc:288] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:21.665822 12788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:21.666378 12788 raft_consensus.cc:686] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [term 1 LEADER]: Becoming Leader. State: Replica: c852c178613a414289c222fd7e51b0d9, State: Running, Role: LEADER
I20260808 11:49:21.667172 12788 consensus_queue.cc:237] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c852c178613a414289c222fd7e51b0d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42719 } }
I20260808 11:49:21.669574 12788 sys_catalog.cc:455] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c852c178613a414289c222fd7e51b0d9. Latest consensus state: current_term: 1 leader_uuid: "c852c178613a414289c222fd7e51b0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c852c178613a414289c222fd7e51b0d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42719 } } }
I20260808 11:49:21.670252 12788 sys_catalog.cc:458] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:21.670778 12791 sys_catalog.cc:455] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c852c178613a414289c222fd7e51b0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c852c178613a414289c222fd7e51b0d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42719 } } }
I20260808 11:49:21.671773 12791 sys_catalog.cc:458] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:21.672729 12777 sys_catalog.cc:564] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:21.676406 12794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:21.677891 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:21.661338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:53103 (local address 127.0.0.1:42719)
0808 11:49:21.661517 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:21.661525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:21.662077 (+ 552us) server_negotiation.cc:407] Connection header received
0808 11:49:21.662272 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:21.662280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:21.662365 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:21.662482 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:21.663977 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:21.664781 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:21.666083 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:21.666784 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:21.666945 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:21.669852 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:21.669858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:21.669864 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:21.669906 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:21.674383 (+ 4477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:21.675472 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:21.675476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:21.675478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:21.675654 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:21.676215 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:21.676220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:21.676223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:21.676606 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:21.676941 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:21.677287 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 11:49:21.677499 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 11:49:21.683063 12794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:21.693559 12794 catalog_manager.cc:1347] Generated new cluster ID: 3db5c7f22772462381443d6bd352bf0f
I20260808 11:49:21.693687 12794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:21.703789 12794 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:21.704895 12794 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:21.711386 12794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c852c178613a414289c222fd7e51b0d9: Generated new TSK 0
I20260808 11:49:21.712145 12794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:21.721626 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 12672
I20260808 11:49:21.722947 12770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:21.809899 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 12672
W20260808 11:49:21.835884 12654 connection.cc:535] client connection to 127.0.0.1:42719 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42719): Connection reset by peer (error 104)
2026-08-08T11:49:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2221 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:21 dist-test-slave-39l8 krb5kdc[12821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:21 dist-test-slave-39l8 krb5kdc[12821](info): set up 2 sockets
Aug 08 11:49:21 dist-test-slave-39l8 krb5kdc[12821](info): commencing operation
krb5kdc: starting...
W20260808 11:49:23.116458 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:23 dist-test-slave-39l8 krb5kdc[12821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:23.417749 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:23.757054 12841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:23.757366 12841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:23.757545 12841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:23.764376 12841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:23.764468 12841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:23.764510 12841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:23.764587 12841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:23.764663 12841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:23.773861 12841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:23.775117 12841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:23.776383 12841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:23.787936 12850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:23.788177 12853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:23.788435 12849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:23.788761 12841 server_base.cc:1034] running on GCE node
I20260808 11:49:23.789552 12841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:23.790879 12841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:23.792089 12841 hybrid_clock.cc:648] HybridClock initialized: now 1786189763792014 us; error 59 us; skew 500 ppm
I20260808 11:49:23.792415 12841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:23 dist-test-slave-39l8 krb5kdc[12821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189763, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:23.795481 12841 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:23.796761 12841 webserver.cc:460] Webserver started at http://127.0.0.1:46741/ using document root <none> and password file <none>
I20260808 11:49:23.797398 12841 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:23.797550 12841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:23.797829 12841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:23.800731 12841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "664628abb3d7462ab7713aa70bc011d2"
format_stamp: "Formatted at 2026-08-08 11:49:23 on dist-test-slave-39l8"
server_key: "0637eab441c12ce1c045783c6dbf50f2"
server_key_iv: "04c483048dfc4643ff8006207d3a53de"
server_key_version: "encryptionkey@0"
I20260808 11:49:23.801426 12841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "664628abb3d7462ab7713aa70bc011d2"
format_stamp: "Formatted at 2026-08-08 11:49:23 on dist-test-slave-39l8"
server_key: "0637eab441c12ce1c045783c6dbf50f2"
server_key_iv: "04c483048dfc4643ff8006207d3a53de"
server_key_version: "encryptionkey@0"
I20260808 11:49:23.806340 12841 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:49:23.809688 12863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:23.811056 12841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:49:23.811228 12841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "664628abb3d7462ab7713aa70bc011d2"
format_stamp: "Formatted at 2026-08-08 11:49:23 on dist-test-slave-39l8"
server_key: "0637eab441c12ce1c045783c6dbf50f2"
server_key_iv: "04c483048dfc4643ff8006207d3a53de"
server_key_version: "encryptionkey@0"
I20260808 11:49:23.811405 12841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:23.823724 12841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:23.827559 12841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:23.827827 12841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:23.846630 12841 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42491
I20260808 11:49:23.846668 12936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42491 every 8 connection(s)
I20260808 11:49:23.847903 12841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:23.848567 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 12841
I20260808 11:49:23.848763 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:23.849393 13945 external_mini_cluster.cc:1428] Setting key 2c1dc09e6beb06cbea6f521647957ad8
I20260808 11:49:23.852267 12938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:23.862373 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:23.851235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:60709 (local address 127.0.0.1:42491)
0808 11:49:23.851993 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:23.852014 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:23.852071 (+ 57us) server_negotiation.cc:407] Connection header received
0808 11:49:23.853702 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:23.853724 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:23.854059 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:23.854406 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:23.855047 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:23.856333 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:23.858055 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:23.858693 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:23.858889 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:23.860210 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:23.860217 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:23.860229 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:23.860723 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:23.860795 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:23.860821 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:23.860941 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:23.861271 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 11:49:23.861452 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":348,"threads_started":1}
I20260808 11:49:23.864409 12938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2: Bootstrap starting.
I20260808 11:49:23.866094 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:23.867687 12938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:23.868638 12938 log.cc:824] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:23.871701 12938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2: No bootstrap required, opened a new log
I20260808 11:49:23.877315 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:23.870555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:48939 (local address 127.0.0.1:42491)
0808 11:49:23.870821 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:23.870830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:23.871271 (+ 441us) server_negotiation.cc:407] Connection header received
0808 11:49:23.871426 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:23.871429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:23.871482 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:23.871555 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:23.872171 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:23.872912 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:23.874785 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:23.875249 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:23.875343 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:23.876528 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:23.876533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:23.876536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:23.876551 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:23.876589 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:23.876606 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:23.876688 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:23.876913 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 11:49:23.876988 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:49:23.880882 12938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "664628abb3d7462ab7713aa70bc011d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } }
I20260808 11:49:23.881232 12938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:23.881346 12938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 664628abb3d7462ab7713aa70bc011d2, State: Initialized, Role: FOLLOWER
I20260808 11:49:23.881888 12938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "664628abb3d7462ab7713aa70bc011d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } }
I20260808 11:49:23.882104 12938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:23.882254 12938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:23.882449 12938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:23.884778 12938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "664628abb3d7462ab7713aa70bc011d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } }
I20260808 11:49:23.885284 12938 leader_election.cc:302] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 664628abb3d7462ab7713aa70bc011d2; no voters:
I20260808 11:49:23.885885 12938 leader_election.cc:288] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:23.885972 12959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:23.887300 12959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [term 1 LEADER]: Becoming Leader. State: Replica: 664628abb3d7462ab7713aa70bc011d2, State: Running, Role: LEADER
I20260808 11:49:23.887861 12959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "664628abb3d7462ab7713aa70bc011d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } }
I20260808 11:49:23.887998 12938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:23.890786 12960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "664628abb3d7462ab7713aa70bc011d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "664628abb3d7462ab7713aa70bc011d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } } }
I20260808 11:49:23.890799 12961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 664628abb3d7462ab7713aa70bc011d2. Latest consensus state: current_term: 1 leader_uuid: "664628abb3d7462ab7713aa70bc011d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "664628abb3d7462ab7713aa70bc011d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42491 } } }
I20260808 11:49:23.891234 12961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:23.891213 12960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:23.893096 12967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:23.897955 12967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:23.906834 12967 catalog_manager.cc:1347] Generated new cluster ID: 684eefe39a8d438f9e7fab2e48158559
I20260808 11:49:23.906934 12967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:23.938030 12967 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:23.939989 12967 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:23.947755 12967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 664628abb3d7462ab7713aa70bc011d2: Generated new TSK 0
I20260808 11:49:23.948762 12967 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:23.956918 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 12841
W20260808 11:49:23.957964 12931 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:60709
W20260808 11:49:23.958945 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 12841: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:23.959102 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 12841
W20260808 11:49:23.983228 12817 connection.cc:535] client connection to 127.0.0.1:42491 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42491): Connection reset by peer (error 104)
2026-08-08T11:49:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:24 dist-test-slave-39l8 krb5kdc[12992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:24 dist-test-slave-39l8 krb5kdc[12992](info): set up 2 sockets
Aug 08 11:49:24 dist-test-slave-39l8 krb5kdc[12992](info): commencing operation
krb5kdc: starting...
W20260808 11:49:25.280922 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.005s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:25 dist-test-slave-39l8 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:49:25.603974 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:25.938807 13010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:25.939102 13010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:25.939196 13010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:25.946386 13010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:25.946492 13010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:25.946538 13010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:25.946590 13010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:25.946626 13010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:25.956125 13010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:25.957363 13010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:25.958701 13010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:25.970598 13022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:25.970690 13019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:25.970659 13020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:25.971586 13010 server_base.cc:1034] running on GCE node
I20260808 11:49:25.972470 13010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:25.973948 13010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:25.975147 13010 hybrid_clock.cc:648] HybridClock initialized: now 1786189765975110 us; error 37 us; skew 500 ppm
I20260808 11:49:25.975489 13010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:25 dist-test-slave-39l8 krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189765, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:25.978766 13010 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:49:25.980194 13010 webserver.cc:460] Webserver started at http://127.0.0.1:34467/ using document root <none> and password file <none>
I20260808 11:49:25.980878 13010 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:25.981029 13010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:25.981336 13010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:25.984373 13010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "19e1f568efb7437cb434abc2738f9f32"
format_stamp: "Formatted at 2026-08-08 11:49:25 on dist-test-slave-39l8"
server_key: "7e576d29df7ca35d908169ac99807be8"
server_key_iv: "b278c69cbd826a86466de1fcd7379201"
server_key_version: "encryptionkey@0"
I20260808 11:49:25.985190 13010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "19e1f568efb7437cb434abc2738f9f32"
format_stamp: "Formatted at 2026-08-08 11:49:25 on dist-test-slave-39l8"
server_key: "7e576d29df7ca35d908169ac99807be8"
server_key_iv: "b278c69cbd826a86466de1fcd7379201"
server_key_version: "encryptionkey@0"
I20260808 11:49:25.990207 13010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:49:25.993686 13031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:25.995023 13010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:49:25.995213 13010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "19e1f568efb7437cb434abc2738f9f32"
format_stamp: "Formatted at 2026-08-08 11:49:25 on dist-test-slave-39l8"
server_key: "7e576d29df7ca35d908169ac99807be8"
server_key_iv: "b278c69cbd826a86466de1fcd7379201"
server_key_version: "encryptionkey@0"
I20260808 11:49:25.995436 13010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:26.001061 13010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:26.005411 13010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:26.005704 13010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:26.032811 13010 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36823
I20260808 11:49:26.032862 13111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36823 every 8 connection(s)
I20260808 11:49:26.034296 13010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:26.038173 13112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:26.044075 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 13010
I20260808 11:49:26.044252 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:26.044800 13945 external_mini_cluster.cc:1428] Setting key 547d4703f5568977baab4386b3aa51c2
I20260808 11:49:26.050325 13112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32: Bootstrap starting.
I20260808 11:49:26.053905 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:26.046740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:44879 (local address 127.0.0.1:36823)
0808 11:49:26.047676 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:26.047691 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:26.047751 (+ 60us) server_negotiation.cc:407] Connection header received
0808 11:49:26.050089 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:26.050116 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:26.050571 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:26.051007 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:26.051939 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:26.051948 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:26.051964 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:26.052009 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:26.052108 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:26.052145 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:26.052380 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:26.052520 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 11:49:26.052723 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":412,"threads_started":1}
I20260808 11:49:26.054375 13112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:26.055619 13112 log.cc:824] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:26.057775 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:26.058830 13112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32: No bootstrap required, opened a new log
I20260808 11:49:26.065217 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:26.062517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:52403 (local address 127.0.0.1:36823)
0808 11:49:26.062765 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:26.062775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:26.063218 (+ 443us) server_negotiation.cc:407] Connection header received
0808 11:49:26.063407 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:26.063410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:26.063458 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:26.063536 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:26.064547 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:26.064551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:26.064554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:26.064572 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:26.064607 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:26.064621 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:26.064723 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:26.064843 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 11:49:26.064895 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:49:26.067800 13112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e1f568efb7437cb434abc2738f9f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } }
I20260808 11:49:26.068188 13112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:26.068317 13112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e1f568efb7437cb434abc2738f9f32, State: Initialized, Role: FOLLOWER
I20260808 11:49:26.068945 13112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e1f568efb7437cb434abc2738f9f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } }
I20260808 11:49:26.069186 13112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:26.069316 13112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:26.069521 13112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:26.071841 13112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e1f568efb7437cb434abc2738f9f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } }
I20260808 11:49:26.072450 13112 leader_election.cc:302] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19e1f568efb7437cb434abc2738f9f32; no voters:
I20260808 11:49:26.073236 13112 leader_election.cc:288] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:26.073367 13126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:26.074117 13126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [term 1 LEADER]: Becoming Leader. State: Replica: 19e1f568efb7437cb434abc2738f9f32, State: Running, Role: LEADER
I20260808 11:49:26.074697 13126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e1f568efb7437cb434abc2738f9f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } }
I20260808 11:49:26.075541 13112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:26.077603 13128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19e1f568efb7437cb434abc2738f9f32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e1f568efb7437cb434abc2738f9f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } } }
I20260808 11:49:26.078009 13128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:26.078898 13129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19e1f568efb7437cb434abc2738f9f32. Latest consensus state: current_term: 1 leader_uuid: "19e1f568efb7437cb434abc2738f9f32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e1f568efb7437cb434abc2738f9f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } } }
I20260808 11:49:26.079358 13129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:26.080888 13136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:26.085774 13136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:26.094476 13136 catalog_manager.cc:1347] Generated new cluster ID: 05cc538abe674ae1b101f787478a51c5
I20260808 11:49:26.094600 13136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:26.109767 13136 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:26.110256 13136 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:26.122767 13136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19e1f568efb7437cb434abc2738f9f32: Generated new TSK 0
I20260808 11:49:26.123778 13136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:26.140724 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13010
W20260808 11:49:26.142091 13101 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:44879
W20260808 11:49:26.142925 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13010: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:26.143081 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 13010
2026-08-08T11:49:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2183 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:26 dist-test-slave-39l8 krb5kdc[13160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:26 dist-test-slave-39l8 krb5kdc[13160](info): set up 2 sockets
Aug 08 11:49:26 dist-test-slave-39l8 krb5kdc[13160](info): commencing operation
krb5kdc: starting...
W20260808 11:49:27.430341 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:27 dist-test-slave-39l8 krb5kdc[13160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:49:27.745453 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:37971
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:41785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:37971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:28.087699 13181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:28.087970 13181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:28.088063 13181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:28.095044 13181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:28.095144 13181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:28.095196 13181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:28.095248 13181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:28.095290 13181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:28.104465 13181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:41785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:37971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:37971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:28.105747 13181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:28.107054 13181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:28.118777 13198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:28.118788 13193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:28.119091 13191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:28.119781 13181 server_base.cc:1034] running on GCE node
I20260808 11:49:28.120731 13181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:28.122125 13181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:28.123370 13181 hybrid_clock.cc:648] HybridClock initialized: now 1786189768123328 us; error 57 us; skew 500 ppm
I20260808 11:49:28.123718 13181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:28 dist-test-slave-39l8 krb5kdc[13160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189768, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:28.131335 13181 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:49:28.132797 13181 webserver.cc:460] Webserver started at http://127.13.158.126:44273/ using document root <none> and password file <none>
I20260808 11:49:28.133502 13181 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:28.133647 13181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:28.133963 13181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:28.136885 13181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "d246b6098baf4dc98300ee5d5ff6c7a6"
format_stamp: "Formatted at 2026-08-08 11:49:28 on dist-test-slave-39l8"
server_key: "fb91a09851f130ee347ee138b326ca3e"
server_key_iv: "9e4b2c0d91c9fe80e447e302b62168fa"
server_key_version: "encryptionkey@0"
I20260808 11:49:28.137655 13181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "d246b6098baf4dc98300ee5d5ff6c7a6"
format_stamp: "Formatted at 2026-08-08 11:49:28 on dist-test-slave-39l8"
server_key: "fb91a09851f130ee347ee138b326ca3e"
server_key_iv: "9e4b2c0d91c9fe80e447e302b62168fa"
server_key_version: "encryptionkey@0"
I20260808 11:49:28.142668 13181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:49:28.146061 13210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:28.147347 13181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:28.147539 13181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "d246b6098baf4dc98300ee5d5ff6c7a6"
format_stamp: "Formatted at 2026-08-08 11:49:28 on dist-test-slave-39l8"
server_key: "fb91a09851f130ee347ee138b326ca3e"
server_key_iv: "9e4b2c0d91c9fe80e447e302b62168fa"
server_key_version: "encryptionkey@0"
I20260808 11:49:28.147768 13181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:28.160033 13181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:28.164057 13181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:28.164332 13181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:28.182947 13181 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:37971
I20260808 11:49:28.182957 13295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:37971 every 8 connection(s)
I20260808 11:49:28.184338 13181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:28.186000 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 13181
I20260808 11:49:28.186187 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:28.186775 13945 external_mini_cluster.cc:1428] Setting key d1bb8ab27bdb1ac41e54cb12990ce014
I20260808 11:49:28.188625 13296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:49:28 dist-test-slave-39l8 krb5kdc[13160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:49:28.202248 13296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6: Bootstrap starting.
I20260808 11:49:28.205483 13296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:28.206534 13296 log.cc:824] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:28.209784 13296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6: No bootstrap required, opened a new log
I20260808 11:49:28.210632 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:28.188794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43261 (local address 127.13.158.126:37971)
0808 11:49:28.189999 (+ 1205us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:28.190013 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:28.190079 (+ 66us) server_negotiation.cc:407] Connection header received
0808 11:49:28.192068 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:28.192098 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:28.192584 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:28.193011 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:28.194326 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:28.196441 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:28.197644 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:28.198688 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:28.198906 (+ 218us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:28.203267 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:28.203275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:28.203287 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:28.203325 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:28.206276 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:28.207290 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:28.207302 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:28.207310 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:28.207533 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:28.208010 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:28.208013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:28.208015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:28.208491 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:28.208743 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:28.209262 (+ 519us) server_negotiation.cc:299] Negotiation successful
0808 11:49:28.209528 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":702,"threads_started":1}
I20260808 11:49:28.214344 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:28.219259 13296 raft_consensus.cc:348] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246b6098baf4dc98300ee5d5ff6c7a6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37971 } }
I20260808 11:49:28.219759 13296 raft_consensus.cc:374] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:28.219956 13296 raft_consensus.cc:729] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d246b6098baf4dc98300ee5d5ff6c7a6, State: Initialized, Role: FOLLOWER
I20260808 11:49:28.220650 13296 consensus_queue.cc:260] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246b6098baf4dc98300ee5d5ff6c7a6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37971 } }
I20260808 11:49:28.220953 13296 raft_consensus.cc:388] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:28.221102 13296 raft_consensus.cc:482] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:28.221305 13296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:28.224323 13296 raft_consensus.cc:504] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246b6098baf4dc98300ee5d5ff6c7a6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37971 } }
I20260808 11:49:28.224901 13296 leader_election.cc:302] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d246b6098baf4dc98300ee5d5ff6c7a6; no voters:
I20260808 11:49:28.225644 13296 leader_election.cc:288] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:28.225783 13311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:28.226473 13311 raft_consensus.cc:686] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [term 1 LEADER]: Becoming Leader. State: Replica: d246b6098baf4dc98300ee5d5ff6c7a6, State: Running, Role: LEADER
I20260808 11:49:28.227314 13311 consensus_queue.cc:237] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246b6098baf4dc98300ee5d5ff6c7a6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37971 } }
I20260808 11:49:28.229625 13312 sys_catalog.cc:455] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d246b6098baf4dc98300ee5d5ff6c7a6. Latest consensus state: current_term: 1 leader_uuid: "d246b6098baf4dc98300ee5d5ff6c7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246b6098baf4dc98300ee5d5ff6c7a6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37971 } } }
I20260808 11:49:28.230034 13312 sys_catalog.cc:458] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:28.230420 13313 sys_catalog.cc:455] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d246b6098baf4dc98300ee5d5ff6c7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246b6098baf4dc98300ee5d5ff6c7a6" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 37971 } } }
I20260808 11:49:28.230641 13296 sys_catalog.cc:564] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:28.230845 13313 sys_catalog.cc:458] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:28.233505 13315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:28.237829 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:28.219832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56183 (local address 127.13.158.126:37971)
0808 11:49:28.219987 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:28.219991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:28.220353 (+ 362us) server_negotiation.cc:407] Connection header received
0808 11:49:28.220515 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:28.220519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:28.220577 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:28.220654 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:28.222678 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:28.223513 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:28.225215 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:28.225712 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:28.225870 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:28.228586 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:28.228594 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:28.228601 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:28.228648 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:28.233592 (+ 4944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:28.234354 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:28.234359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:28.234363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:28.234504 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:28.234999 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:28.235007 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:28.235010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:28.235493 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:28.237008 (+ 1515us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:28.237196 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 11:49:28.237396 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 11:49:28.239542 13315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:28.250504 13315 catalog_manager.cc:1347] Generated new cluster ID: c426d44c30284fe5b4f662f797bd22d8
I20260808 11:49:28.250618 13315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:28.262863 13315 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:28.264249 13315 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:28.270342 13315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d246b6098baf4dc98300ee5d5ff6c7a6: Generated new TSK 0
I20260808 11:49:28.271237 13315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:28.281232 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13181
I20260808 11:49:28.282998 13284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:28.372560 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 13181
W20260808 11:49:28.398531 13157 connection.cc:535] client connection to 127.13.158.126:37971 recv error: Network error: failed to read from TLS socket (remote: 127.13.158.126:37971): Connection reset by peer (error 104)
2026-08-08T11:49:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2232 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:28 dist-test-slave-39l8 krb5kdc[13344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:28 dist-test-slave-39l8 krb5kdc[13344](info): set up 2 sockets
Aug 08 11:49:28 dist-test-slave-39l8 krb5kdc[13344](info): commencing operation
krb5kdc: starting...
W20260808 11:49:29.679675 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.001s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:29 dist-test-slave-39l8 krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:49:30.002776 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43589
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:43589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:30.347012 13365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:30.347316 13365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:30.347514 13365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:30.354772 13365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:30.354930 13365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:30.355007 13365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:30.355073 13365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:30.355118 13365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:30.365218 13365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:43589
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:43589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:30.366513 13365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:30.367878 13365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:30.379599 13375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:30.379748 13371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:30.379655 13372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:30.380890 13365 server_base.cc:1034] running on GCE node
I20260808 11:49:30.381634 13365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:30.383004 13365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:30.384232 13365 hybrid_clock.cc:648] HybridClock initialized: now 1786189770384186 us; error 54 us; skew 500 ppm
I20260808 11:49:30.384565 13365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:30 dist-test-slave-39l8 krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189770, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:30.387650 13365 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:49:30.389017 13365 webserver.cc:460] Webserver started at http://127.13.158.126:34467/ using document root <none> and password file <none>
I20260808 11:49:30.389701 13365 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:30.389849 13365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:30.390157 13365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:30.393180 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "4657f12fa9a243af810993576c8925b9"
format_stamp: "Formatted at 2026-08-08 11:49:30 on dist-test-slave-39l8"
server_key: "6d9b7b0959b1499c814c281be3154275"
server_key_iv: "50e81251e118335a44478746775694fe"
server_key_version: "encryptionkey@0"
I20260808 11:49:30.393949 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "4657f12fa9a243af810993576c8925b9"
format_stamp: "Formatted at 2026-08-08 11:49:30 on dist-test-slave-39l8"
server_key: "6d9b7b0959b1499c814c281be3154275"
server_key_iv: "50e81251e118335a44478746775694fe"
server_key_version: "encryptionkey@0"
I20260808 11:49:30.398978 13365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260808 11:49:30.402240 13388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:30.403565 13365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:30.403753 13365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "4657f12fa9a243af810993576c8925b9"
format_stamp: "Formatted at 2026-08-08 11:49:30 on dist-test-slave-39l8"
server_key: "6d9b7b0959b1499c814c281be3154275"
server_key_iv: "50e81251e118335a44478746775694fe"
server_key_version: "encryptionkey@0"
I20260808 11:49:30.403970 13365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:30.416713 13365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:30.420645 13365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:30.420931 13365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:30.439868 13365 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:43589
I20260808 11:49:30.439893 13459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:43589 every 8 connection(s)
I20260808 11:49:30.441301 13365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:30.445091 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:30.447360 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 13365
I20260808 11:49:30.447561 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:30.448002 13945 external_mini_cluster.cc:1428] Setting key 47b15123739b63b6ab660231c93f685f
I20260808 11:49:30.458577 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9: Bootstrap starting.
I20260808 11:49:30.462666 13461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:30.463316 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:30.449843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45379 (local address 127.13.158.126:43589)
0808 11:49:30.450685 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:30.450720 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:30.450775 (+ 55us) server_negotiation.cc:407] Connection header received
0808 11:49:30.453185 (+ 2410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:30.453223 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:30.453690 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:30.454115 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:30.454643 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:30.456819 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:30.458107 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:30.459164 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:30.459351 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:30.460550 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:30.460560 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:30.460577 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:30.461297 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:30.461391 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:30.461422 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:30.461573 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:30.461805 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 11:49:30.461969 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":357,"threads_started":1}
I20260808 11:49:30.463943 13461 log.cc:824] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:30.466933 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9: No bootstrap required, opened a new log
I20260808 11:49:30.467212 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:30.477272 13461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4657f12fa9a243af810993576c8925b9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43589 } }
I20260808 11:49:30.477733 13461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:30.477836 13461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4657f12fa9a243af810993576c8925b9, State: Initialized, Role: FOLLOWER
I20260808 11:49:30.478585 13461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4657f12fa9a243af810993576c8925b9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43589 } }
I20260808 11:49:30.478933 13461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:30.479153 13461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:30.479358 13461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:30.479386 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:30.472516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33557 (local address 127.13.158.126:43589)
0808 11:49:30.472713 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:30.472724 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:30.473153 (+ 429us) server_negotiation.cc:407] Connection header received
0808 11:49:30.473332 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:30.473338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:30.473439 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:30.473559 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:30.474048 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:30.475262 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:30.476564 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:30.477204 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:30.477346 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:30.478504 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:30.478508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:30.478512 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:30.478528 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:30.478565 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:30.478583 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:30.478670 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:30.478973 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 11:49:30.479032 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:49:30.482016 13461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4657f12fa9a243af810993576c8925b9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43589 } }
I20260808 11:49:30.482502 13461 leader_election.cc:302] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4657f12fa9a243af810993576c8925b9; no voters:
I20260808 11:49:30.483112 13461 leader_election.cc:288] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:30.483244 13478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:30.483911 13478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [term 1 LEADER]: Becoming Leader. State: Replica: 4657f12fa9a243af810993576c8925b9, State: Running, Role: LEADER
I20260808 11:49:30.484417 13478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4657f12fa9a243af810993576c8925b9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43589 } }
I20260808 11:49:30.485194 13461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:30.487721 13479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4657f12fa9a243af810993576c8925b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4657f12fa9a243af810993576c8925b9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43589 } } }
I20260808 11:49:30.487798 13480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4657f12fa9a243af810993576c8925b9. Latest consensus state: current_term: 1 leader_uuid: "4657f12fa9a243af810993576c8925b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4657f12fa9a243af810993576c8925b9" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 43589 } } }
I20260808 11:49:30.488142 13479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:30.488215 13480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:30.495469 13489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:30.500736 13489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:30.509362 13489 catalog_manager.cc:1347] Generated new cluster ID: 69ff10ef0ff242bc909ac2964fce0ca5
I20260808 11:49:30.509474 13489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:30.519799 13489 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:30.521399 13489 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:30.529451 13489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4657f12fa9a243af810993576c8925b9: Generated new TSK 0
I20260808 11:49:30.530491 13489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:30.541342 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13365
W20260808 11:49:30.542517 13454 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45379
W20260808 11:49:30.543354 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13365: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:30.543503 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 13365
W20260808 11:49:30.570647 13339 connection.cc:535] client connection to 127.13.158.126:43589 recv error: Network error: failed to read from TLS socket (remote: 127.13.158.126:43589): Connection reset by peer (error 104)
2026-08-08T11:49:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2171 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:30 dist-test-slave-39l8 krb5kdc[13512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:30 dist-test-slave-39l8 krb5kdc[13512](info): set up 2 sockets
Aug 08 11:49:30 dist-test-slave-39l8 krb5kdc[13512](info): commencing operation
krb5kdc: starting...
W20260808 11:49:31.831046 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:32 dist-test-slave-39l8 krb5kdc[13512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:49:32.148105 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44683
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:44683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:32.486563 13530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:32.486917 13530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:32.487099 13530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:32.494205 13530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:32.494349 13530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:32.494454 13530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:32.494599 13530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:32.494689 13530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:32.504446 13530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:44683
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:32.505738 13530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:32.507016 13530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:32.519379 13543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:32.519340 13545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:32.519387 13548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:32.520218 13530 server_base.cc:1034] running on GCE node
I20260808 11:49:32.520949 13530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:32.522238 13530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:32.523453 13530 hybrid_clock.cc:648] HybridClock initialized: now 1786189772523419 us; error 31 us; skew 500 ppm
I20260808 11:49:32.523782 13530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:32 dist-test-slave-39l8 krb5kdc[13512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189772, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:32.526973 13530 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:49:32.528432 13530 webserver.cc:460] Webserver started at http://127.13.158.126:44239/ using document root <none> and password file <none>
I20260808 11:49:32.529101 13530 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:32.529253 13530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:32.529556 13530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:32.532507 13530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "d4e53827a62e42aa8b6ae52b68f508d3"
format_stamp: "Formatted at 2026-08-08 11:49:32 on dist-test-slave-39l8"
server_key: "ad640a8b24636019bd71d11cdce202e4"
server_key_iv: "e5614abfe9c97e4242859a37636af9a0"
server_key_version: "encryptionkey@0"
I20260808 11:49:32.533264 13530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "d4e53827a62e42aa8b6ae52b68f508d3"
format_stamp: "Formatted at 2026-08-08 11:49:32 on dist-test-slave-39l8"
server_key: "ad640a8b24636019bd71d11cdce202e4"
server_key_iv: "e5614abfe9c97e4242859a37636af9a0"
server_key_version: "encryptionkey@0"
I20260808 11:49:32.538249 13530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:49:32.541728 13558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:32.542999 13530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:32.543187 13530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "d4e53827a62e42aa8b6ae52b68f508d3"
format_stamp: "Formatted at 2026-08-08 11:49:32 on dist-test-slave-39l8"
server_key: "ad640a8b24636019bd71d11cdce202e4"
server_key_iv: "e5614abfe9c97e4242859a37636af9a0"
server_key_version: "encryptionkey@0"
I20260808 11:49:32.543424 13530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:32.548987 13530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:32.552958 13530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:32.553267 13530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:32.571558 13530 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:44683
I20260808 11:49:32.571587 13643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:44683 every 8 connection(s)
I20260808 11:49:32.572991 13530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:32.576751 13644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:32.580822 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 13530
I20260808 11:49:32.581019 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:32.581575 13945 external_mini_cluster.cc:1428] Setting key 874e20a10e494a33975bfb36f6c828ce
I20260808 11:49:32.589074 13649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:32.583370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42961 (local address 127.13.158.126:44683)
0808 11:49:32.584158 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:32.584169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:32.584227 (+ 58us) server_negotiation.cc:407] Connection header received
0808 11:49:32.585821 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:32.585844 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:32.586185 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:32.586511 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:32.587546 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:32.587555 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:32.587569 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:32.587588 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:32.587639 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:32.587664 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:32.587866 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:32.587961 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 11:49:32.588204 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":383,"threads_started":1}
I20260808 11:49:32.589819 13644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3: Bootstrap starting.
I20260808 11:49:32.592737 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:32.593029 13644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:32.593998 13644 log.cc:824] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:32.596995 13644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3: No bootstrap required, opened a new log
I20260808 11:49:32.601163 13649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:32.597134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43505 (local address 127.13.158.126:44683)
0808 11:49:32.597322 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:32.597329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:32.599307 (+ 1978us) server_negotiation.cc:407] Connection header received
0808 11:49:32.599421 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:32.599428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:32.599498 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:32.599607 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:32.600544 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:32.600549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:32.600553 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:32.600570 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:32.600617 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:32.600636 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:32.600746 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:32.600832 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 11:49:32.600881 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:49:32.605461 13644 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e53827a62e42aa8b6ae52b68f508d3" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44683 } }
I20260808 11:49:32.605831 13644 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:32.605955 13644 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e53827a62e42aa8b6ae52b68f508d3, State: Initialized, Role: FOLLOWER
I20260808 11:49:32.606535 13644 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e53827a62e42aa8b6ae52b68f508d3" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44683 } }
I20260808 11:49:32.606803 13644 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:32.606949 13644 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:32.607108 13644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:32.609160 13644 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e53827a62e42aa8b6ae52b68f508d3" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44683 } }
I20260808 11:49:32.609632 13644 leader_election.cc:302] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4e53827a62e42aa8b6ae52b68f508d3; no voters:
I20260808 11:49:32.610258 13644 leader_election.cc:288] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:32.610392 13661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:32.611348 13661 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [term 1 LEADER]: Becoming Leader. State: Replica: d4e53827a62e42aa8b6ae52b68f508d3, State: Running, Role: LEADER
I20260808 11:49:32.611922 13661 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e53827a62e42aa8b6ae52b68f508d3" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44683 } }
I20260808 11:49:32.612615 13644 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:32.615073 13663 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4e53827a62e42aa8b6ae52b68f508d3. Latest consensus state: current_term: 1 leader_uuid: "d4e53827a62e42aa8b6ae52b68f508d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e53827a62e42aa8b6ae52b68f508d3" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44683 } } }
I20260808 11:49:32.615558 13663 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:32.615499 13662 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4e53827a62e42aa8b6ae52b68f508d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e53827a62e42aa8b6ae52b68f508d3" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44683 } } }
I20260808 11:49:32.615890 13662 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:32.618311 13671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:32.623415 13671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:32.631587 13671 catalog_manager.cc:1347] Generated new cluster ID: 38bc44030c404b518ba4677bd49c4836
I20260808 11:49:32.631716 13671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:32.650072 13671 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:32.650470 13671 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:32.660100 13671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4e53827a62e42aa8b6ae52b68f508d3: Generated new TSK 0
I20260808 11:49:32.660907 13671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:32.664927 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13530
W20260808 11:49:32.666035 13632 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42961
W20260808 11:49:32.666697 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13530: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:32.666868 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 13530
2026-08-08T11:49:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2123 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:32 dist-test-slave-39l8 krb5kdc[13687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:32 dist-test-slave-39l8 krb5kdc[13687](info): set up 2 sockets
Aug 08 11:49:32 dist-test-slave-39l8 krb5kdc[13687](info): commencing operation
krb5kdc: starting...
W20260808 11:49:33.968807 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.007s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:34 dist-test-slave-39l8 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:49:34.308173 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:34635
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:34635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:34.645856 13708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:34.646130 13708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:34.646224 13708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:34.653371 13708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:34.653474 13708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:34.653525 13708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:34.653606 13708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:34.653695 13708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:34.663085 13708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:34635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:34635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:34.664297 13708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:34.665609 13708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:34.678006 13714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:34.678038 13719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:34.678108 13716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:34.680253 13708 server_base.cc:1034] running on GCE node
I20260808 11:49:34.680999 13708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:34.682369 13708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:34.683596 13708 hybrid_clock.cc:648] HybridClock initialized: now 1786189774683551 us; error 55 us; skew 500 ppm
I20260808 11:49:34.683931 13708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:34 dist-test-slave-39l8 krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189774, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:34.691640 13708 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:49:34.693308 13708 webserver.cc:460] Webserver started at http://127.13.158.126:43747/ using document root <none> and password file <none>
I20260808 11:49:34.694065 13708 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:34.694218 13708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:34.694585 13708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:34.697679 13708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "99dd66577bc041eeb76370731a2fc89c"
format_stamp: "Formatted at 2026-08-08 11:49:34 on dist-test-slave-39l8"
server_key: "e18841e1ab586a39e2cd3540fd3d45e0"
server_key_iv: "2a74e52410f9c8cc1134acf6eb4455c1"
server_key_version: "encryptionkey@0"
I20260808 11:49:34.698468 13708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "99dd66577bc041eeb76370731a2fc89c"
format_stamp: "Formatted at 2026-08-08 11:49:34 on dist-test-slave-39l8"
server_key: "e18841e1ab586a39e2cd3540fd3d45e0"
server_key_iv: "2a74e52410f9c8cc1134acf6eb4455c1"
server_key_version: "encryptionkey@0"
I20260808 11:49:34.703939 13708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:49:34.707484 13735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:34.708772 13708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:49:34.708959 13708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "99dd66577bc041eeb76370731a2fc89c"
format_stamp: "Formatted at 2026-08-08 11:49:34 on dist-test-slave-39l8"
server_key: "e18841e1ab586a39e2cd3540fd3d45e0"
server_key_iv: "2a74e52410f9c8cc1134acf6eb4455c1"
server_key_version: "encryptionkey@0"
I20260808 11:49:34.709175 13708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:34.740221 13708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:34.744241 13708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:34.744522 13708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:34.763352 13708 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:34635
I20260808 11:49:34.763382 13806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:34635 every 8 connection(s)
I20260808 11:49:34.764716 13708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:34.768556 13808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:34.772464 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 13708
I20260808 11:49:34.772660 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:34.773221 13945 external_mini_cluster.cc:1428] Setting key cba26bcb81724013c8e71f6ad7176fca
I20260808 11:49:34.781751 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c: Bootstrap starting.
I20260808 11:49:34.785678 13808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:34.787096 13808 log.cc:824] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:49:34 dist-test-slave-39l8 krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:49:34.790060 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c: No bootstrap required, opened a new log
I20260808 11:49:34.796758 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:34.775157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56829 (local address 127.13.158.126:34635)
0808 11:49:34.775997 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:34.776008 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:34.776048 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:49:34.777638 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:34.777659 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:34.778045 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:34.778361 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:34.780166 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:34.781510 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:34.783328 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:34.783991 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:34.784123 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:34.789292 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:34.789300 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:34.789314 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:34.789348 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:34.792604 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:34.793304 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:34.793310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:34.793316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:34.793482 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:34.793893 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:34.793897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:34.793899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:34.794474 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:34.794858 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:34.795326 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 11:49:34.795739 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":430,"threads_started":1}
I20260808 11:49:34.798107 13808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99dd66577bc041eeb76370731a2fc89c" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34635 } }
I20260808 11:49:34.798555 13808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:34.798676 13808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dd66577bc041eeb76370731a2fc89c, State: Initialized, Role: FOLLOWER
I20260808 11:49:34.799336 13808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99dd66577bc041eeb76370731a2fc89c" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34635 } }
I20260808 11:49:34.799566 13808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:34.799721 13808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:34.799929 13808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:34.800694 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:34.802331 13808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99dd66577bc041eeb76370731a2fc89c" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34635 } }
I20260808 11:49:34.802912 13808 leader_election.cc:302] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99dd66577bc041eeb76370731a2fc89c; no voters:
I20260808 11:49:34.803541 13808 leader_election.cc:288] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:34.803670 13818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:34.806007 13808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:34.807001 13818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [term 1 LEADER]: Becoming Leader. State: Replica: 99dd66577bc041eeb76370731a2fc89c, State: Running, Role: LEADER
I20260808 11:49:34.807695 13818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99dd66577bc041eeb76370731a2fc89c" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34635 } }
I20260808 11:49:34.811398 13822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99dd66577bc041eeb76370731a2fc89c. Latest consensus state: current_term: 1 leader_uuid: "99dd66577bc041eeb76370731a2fc89c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99dd66577bc041eeb76370731a2fc89c" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34635 } } }
I20260808 11:49:34.811866 13822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:34.812413 13821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99dd66577bc041eeb76370731a2fc89c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99dd66577bc041eeb76370731a2fc89c" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 34635 } } }
I20260808 11:49:34.812770 13821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:34.824424 13831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:34.835300 13831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:34.838675 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:34.811899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44181 (local address 127.13.158.126:34635)
0808 11:49:34.812115 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:34.812123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:34.822678 (+ 10555us) server_negotiation.cc:407] Connection header received
0808 11:49:34.822970 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:34.822975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:34.823041 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:34.823108 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:34.824952 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:34.825708 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:34.827217 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:34.827785 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:34.827858 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:34.830574 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:34.830585 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:34.830588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:34.830619 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:34.834619 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:34.835520 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:34.835526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:34.835531 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:34.835702 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:34.836150 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:34.836160 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:34.836164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:34.836621 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:34.836887 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:34.838055 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0808 11:49:34.838303 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:49:34.846354 13831 catalog_manager.cc:1347] Generated new cluster ID: c1887142f456489cb400e05bee7a19ba
I20260808 11:49:34.846457 13831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:34.857632 13831 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:34.858896 13831 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:34.875787 13831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99dd66577bc041eeb76370731a2fc89c: Generated new TSK 0
I20260808 11:49:34.876767 13831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:34.887955 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13708
I20260808 11:49:34.889066 13802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:34.980563 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 13708
2026-08-08T11:49:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:35 dist-test-slave-39l8 krb5kdc[13853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:35 dist-test-slave-39l8 krb5kdc[13853](info): set up 2 sockets
Aug 08 11:49:35 dist-test-slave-39l8 krb5kdc[13853](info): commencing operation
krb5kdc: starting...
W20260808 11:49:36.279172 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:36 dist-test-slave-39l8 krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:49:36.573657 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36839
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:41305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:36839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:36.907300 13871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:36.907601 13871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:36.907766 13871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:36.914657 13871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:36.914824 13871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:36.914920 13871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:36.915031 13871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:36.915115 13871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:36.924415 13871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:41305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:36839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:36839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:36.925653 13871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:36.926872 13871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:36.938516 13878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:36.938582 13876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:36.938756 13881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:36.939380 13871 server_base.cc:1034] running on GCE node
I20260808 11:49:36.940089 13871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:36.941358 13871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:36.942585 13871 hybrid_clock.cc:648] HybridClock initialized: now 1786189776942549 us; error 48 us; skew 500 ppm
I20260808 11:49:36.942935 13871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:36 dist-test-slave-39l8 krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189776, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:36.946102 13871 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:49:36.947569 13871 webserver.cc:460] Webserver started at http://127.13.158.126:46367/ using document root <none> and password file <none>
I20260808 11:49:36.948225 13871 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:36.948369 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:36.948715 13871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:36.951627 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "841c15d67d6c44ad9bc26a667163ae68"
format_stamp: "Formatted at 2026-08-08 11:49:36 on dist-test-slave-39l8"
server_key: "c4e43409bf75f90db6546add7262470b"
server_key_iv: "7e488f2296574d3e5a5540e29e463581"
server_key_version: "encryptionkey@0"
I20260808 11:49:36.952378 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "841c15d67d6c44ad9bc26a667163ae68"
format_stamp: "Formatted at 2026-08-08 11:49:36 on dist-test-slave-39l8"
server_key: "c4e43409bf75f90db6546add7262470b"
server_key_iv: "7e488f2296574d3e5a5540e29e463581"
server_key_version: "encryptionkey@0"
I20260808 11:49:36.957170 13871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:49:36.960548 13890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:36.961723 13871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:49:36.961892 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "841c15d67d6c44ad9bc26a667163ae68"
format_stamp: "Formatted at 2026-08-08 11:49:36 on dist-test-slave-39l8"
server_key: "c4e43409bf75f90db6546add7262470b"
server_key_iv: "7e488f2296574d3e5a5540e29e463581"
server_key_version: "encryptionkey@0"
I20260808 11:49:36.962091 13871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:36.986064 13871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:36.989984 13871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:36.990275 13871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:37.009377 13871 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:36839
I20260808 11:49:37.009400 13984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:36839 every 8 connection(s)
I20260808 11:49:37.010878 13871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:37.014672 13985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:37.019975 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 13871
I20260808 11:49:37.020179 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:37.020644 13945 external_mini_cluster.cc:1428] Setting key eece1e23955fd3279c7e40f758486d21
I20260808 11:49:37.025966 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68: Bootstrap starting.
I20260808 11:49:37.029577 13985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:37.030650 13985 log.cc:824] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:37.033339 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68: No bootstrap required, opened a new log
I20260808 11:49:37.035077 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:37.022541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51345 (local address 127.13.158.126:36839)
0808 11:49:37.023424 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:37.023444 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:37.023494 (+ 50us) server_negotiation.cc:407] Connection header received
0808 11:49:37.025790 (+ 2296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:37.025818 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:37.026284 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:37.026674 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:37.027168 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:37.029008 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:37.030427 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:37.031479 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:37.031718 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:37.032897 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:37.032904 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:37.032918 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:37.033416 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:37.033528 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:37.033555 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:37.033680 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:37.033897 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 11:49:37.034146 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":416,"threads_started":1}
I20260808 11:49:37.038595 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:37.041285 13985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841c15d67d6c44ad9bc26a667163ae68" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36839 } }
I20260808 11:49:37.041687 13985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:37.041805 13985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841c15d67d6c44ad9bc26a667163ae68, State: Initialized, Role: FOLLOWER
I20260808 11:49:37.042328 13985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841c15d67d6c44ad9bc26a667163ae68" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36839 } }
I20260808 11:49:37.042572 13985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:37.042732 13985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:37.042937 13985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:37.045913 13985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841c15d67d6c44ad9bc26a667163ae68" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36839 } }
I20260808 11:49:37.046499 13985 leader_election.cc:302] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 841c15d67d6c44ad9bc26a667163ae68; no voters:
I20260808 11:49:37.047207 13985 leader_election.cc:288] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:37.047314 13999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:37.049185 13999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [term 1 LEADER]: Becoming Leader. State: Replica: 841c15d67d6c44ad9bc26a667163ae68, State: Running, Role: LEADER
I20260808 11:49:37.049827 13999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841c15d67d6c44ad9bc26a667163ae68" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36839 } }
I20260808 11:49:37.050303 13985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:37.050802 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:37.042975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43845 (local address 127.13.158.126:36839)
0808 11:49:37.043175 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:37.043183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:37.043683 (+ 500us) server_negotiation.cc:407] Connection header received
0808 11:49:37.043851 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:37.043858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:37.043912 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:37.043992 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:37.044592 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:37.045236 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:37.047163 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:37.048149 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:37.048313 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:37.049696 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:37.049704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:37.049708 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:37.049728 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:37.049773 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:37.049795 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:37.049902 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:37.050332 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 11:49:37.050460 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:49:37.054334 14000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "841c15d67d6c44ad9bc26a667163ae68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841c15d67d6c44ad9bc26a667163ae68" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36839 } } }
I20260808 11:49:37.055047 14002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 841c15d67d6c44ad9bc26a667163ae68. Latest consensus state: current_term: 1 leader_uuid: "841c15d67d6c44ad9bc26a667163ae68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841c15d67d6c44ad9bc26a667163ae68" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 36839 } } }
I20260808 11:49:37.055335 14002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:37.055694 14000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:37.057317 14020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:37.061957 14020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:37.070900 14020 catalog_manager.cc:1347] Generated new cluster ID: f8fc612c137e41259731660df5eb49ad
I20260808 11:49:37.071010 14020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:37.081084 14020 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:37.082293 14020 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:37.101145 14020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 841c15d67d6c44ad9bc26a667163ae68: Generated new TSK 0
I20260808 11:49:37.101997 14020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:37.114799 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13871
W20260808 11:49:37.115859 13976 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51345
W20260808 11:49:37.116585 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 13871: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:37.116735 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 13871
2026-08-08T11:49:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2136 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:37 dist-test-slave-39l8 krb5kdc[14029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:37 dist-test-slave-39l8 krb5kdc[14029](info): set up 2 sockets
Aug 08 11:49:37 dist-test-slave-39l8 krb5kdc[14029](info): commencing operation
krb5kdc: starting...
W20260808 11:49:38.406988 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.005s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:38 dist-test-slave-39l8 krb5kdc[14029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:49:38.707573 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:42193
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:42193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:39.039646 14048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:39.039973 14048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:39.040163 14048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:39.046921 14048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:39.047061 14048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:39.047180 14048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:39.047294 14048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:39.047384 14048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:39.056488 14048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:42193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:42193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:39.057734 14048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:39.058996 14048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:39.070946 14055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:39.070984 14059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:39.071020 14056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:39.072079 14048 server_base.cc:1034] running on GCE node
I20260808 11:49:39.072831 14048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:39.074127 14048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:39.075429 14048 hybrid_clock.cc:648] HybridClock initialized: now 1786189779075357 us; error 50 us; skew 500 ppm
I20260808 11:49:39.075789 14048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:39 dist-test-slave-39l8 krb5kdc[14029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189779, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:39.078944 14048 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:49:39.080421 14048 webserver.cc:460] Webserver started at http://127.13.158.126:34451/ using document root <none> and password file <none>
I20260808 11:49:39.081115 14048 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:39.081261 14048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:39.081578 14048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:39.084576 14048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "8f7edaf103e74885b6e04317b591bf8a"
format_stamp: "Formatted at 2026-08-08 11:49:39 on dist-test-slave-39l8"
server_key: "d69a24a8ff62e48b6c280e343a2ff728"
server_key_iv: "5bd7b22a8482d046df5e4feac0a77680"
server_key_version: "encryptionkey@0"
I20260808 11:49:39.085320 14048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "8f7edaf103e74885b6e04317b591bf8a"
format_stamp: "Formatted at 2026-08-08 11:49:39 on dist-test-slave-39l8"
server_key: "d69a24a8ff62e48b6c280e343a2ff728"
server_key_iv: "5bd7b22a8482d046df5e4feac0a77680"
server_key_version: "encryptionkey@0"
I20260808 11:49:39.090420 14048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:49:39.093866 14071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:39.095471 14048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:39.095662 14048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "8f7edaf103e74885b6e04317b591bf8a"
format_stamp: "Formatted at 2026-08-08 11:49:39 on dist-test-slave-39l8"
server_key: "d69a24a8ff62e48b6c280e343a2ff728"
server_key_iv: "5bd7b22a8482d046df5e4feac0a77680"
server_key_version: "encryptionkey@0"
I20260808 11:49:39.095911 14048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:39.101620 14048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:39.105504 14048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:39.105798 14048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:39.123872 14048 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:42193
I20260808 11:49:39.123911 14149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:42193 every 8 connection(s)
I20260808 11:49:39.125168 14048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:39.128602 14150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:39.135139 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14048
I20260808 11:49:39.135341 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:39.135924 13945 external_mini_cluster.cc:1428] Setting key fcb00e82d548cea14602241e1005dd02
I20260808 11:49:39.139494 14150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a: Bootstrap starting.
I20260808 11:49:39.142673 14150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:39.143507 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:39.137777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35059 (local address 127.13.158.126:42193)
0808 11:49:39.138522 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:39.138532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:39.138571 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:49:39.140265 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:39.140287 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:39.140591 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:39.140981 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:39.141974 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:39.141980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:39.141992 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:39.142005 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:39.142067 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:39.142085 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:39.142280 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:39.142409 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 11:49:39.142590 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":353,"threads_started":1}
I20260808 11:49:39.143849 14150 log.cc:824] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:39.146865 14150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a: No bootstrap required, opened a new log
I20260808 11:49:39.147149 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:39.154111 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:39.151552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37243 (local address 127.13.158.126:42193)
0808 11:49:39.151776 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:39.151781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:39.152155 (+ 374us) server_negotiation.cc:407] Connection header received
0808 11:49:39.152312 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:39.152315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:39.152360 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:39.152440 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:39.153413 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:39.153417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:39.153420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:49:39.153434 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:39.153479 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:49:39.153494 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:39.153596 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:39.153748 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 11:49:39.153792 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 11:49:39.155669 14150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7edaf103e74885b6e04317b591bf8a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42193 } }
I20260808 11:49:39.156032 14150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:49:39.156143 14150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7edaf103e74885b6e04317b591bf8a, State: Initialized, Role: FOLLOWER
I20260808 11:49:39.156656 14150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7edaf103e74885b6e04317b591bf8a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42193 } }
I20260808 11:49:39.156908 14150 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:49:39.157071 14150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:49:39.157262 14150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:39.159749 14150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7edaf103e74885b6e04317b591bf8a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42193 } }
I20260808 11:49:39.160243 14150 leader_election.cc:302] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f7edaf103e74885b6e04317b591bf8a; no voters:
I20260808 11:49:39.160809 14150 leader_election.cc:288] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:49:39.160945 14162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:39.161744 14162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [term 1 LEADER]: Becoming Leader. State: Replica: 8f7edaf103e74885b6e04317b591bf8a, State: Running, Role: LEADER
I20260808 11:49:39.162294 14162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7edaf103e74885b6e04317b591bf8a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42193 } }
I20260808 11:49:39.162995 14150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:39.164963 14165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f7edaf103e74885b6e04317b591bf8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7edaf103e74885b6e04317b591bf8a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42193 } } }
I20260808 11:49:39.165422 14166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f7edaf103e74885b6e04317b591bf8a. Latest consensus state: current_term: 1 leader_uuid: "8f7edaf103e74885b6e04317b591bf8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7edaf103e74885b6e04317b591bf8a" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 42193 } } }
I20260808 11:49:39.165720 14166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:39.165403 14165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:39.167999 14171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:39.173157 14171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:49:39.180960 14171 catalog_manager.cc:1347] Generated new cluster ID: 76e09e2457e8489990a8214768d721ae
I20260808 11:49:39.181077 14171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:39.195598 14171 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:39.196074 14171 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:39.204831 14171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f7edaf103e74885b6e04317b591bf8a: Generated new TSK 0
I20260808 11:49:39.205593 14171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:39.211045 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14048
W20260808 11:49:39.212275 14145 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35059
W20260808 11:49:39.212971 13945 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14048: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:49:39.213122 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14048
2026-08-08T11:49:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2100 ms)
[----------] 18 tests from AuthTokenIssuingTest (39200 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:39 dist-test-slave-39l8 krb5kdc[14188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:39 dist-test-slave-39l8 krb5kdc[14188](info): set up 2 sockets
Aug 08 11:49:39 dist-test-slave-39l8 krb5kdc[14188](info): commencing operation
krb5kdc: starting...
W20260808 11:49:40.529392 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:40 dist-test-slave-39l8 krb5kdc[14188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:49:40.845774 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44993
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:44993,127.13.158.125:46499,127.13.158.124:35685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:41.179646 14204 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:49:41.180011 14204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:41.180141 14204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:41.180285 14204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:41.187417 14204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:41.187497 14204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:41.187611 14204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:41.187697 14204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:41.187808 14204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:41.197378 14204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:44993,127.13.158.125:46499,127.13.158.124:35685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:44993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:41.198575 14204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:41.199853 14204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:41.211047 14217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:41.211145 14218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:41.211158 14221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:41.211925 14204 server_base.cc:1034] running on GCE node
I20260808 11:49:41.212646 14204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:41.213860 14204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:41.215063 14204 hybrid_clock.cc:648] HybridClock initialized: now 1786189781215013 us; error 33 us; skew 500 ppm
I20260808 11:49:41.215413 14204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:41 dist-test-slave-39l8 krb5kdc[14188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189781, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:41.223167 14204 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:49:41.224754 14204 webserver.cc:460] Webserver started at http://127.13.158.126:36189/ using document root <none> and password file <none>
I20260808 11:49:41.225473 14204 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:41.225592 14204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:41.225986 14204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:41.229120 14204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "7f2a20c9149d48c8b121739620b9ce4f"
format_stamp: "Formatted at 2026-08-08 11:49:41 on dist-test-slave-39l8"
server_key: "f911d6500f8a7be32a7511532702091f"
server_key_iv: "8a7acbe1cafb3e53eeb4e20c306658ca"
server_key_version: "encryptionkey@0"
I20260808 11:49:41.229938 14204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "7f2a20c9149d48c8b121739620b9ce4f"
format_stamp: "Formatted at 2026-08-08 11:49:41 on dist-test-slave-39l8"
server_key: "f911d6500f8a7be32a7511532702091f"
server_key_iv: "8a7acbe1cafb3e53eeb4e20c306658ca"
server_key_version: "encryptionkey@0"
I20260808 11:49:41.235208 14204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:49:41.238677 14232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:41.240037 14204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:49:41.240243 14204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "7f2a20c9149d48c8b121739620b9ce4f"
format_stamp: "Formatted at 2026-08-08 11:49:41 on dist-test-slave-39l8"
server_key: "f911d6500f8a7be32a7511532702091f"
server_key_iv: "8a7acbe1cafb3e53eeb4e20c306658ca"
server_key_version: "encryptionkey@0"
I20260808 11:49:41.240473 14204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:41.253340 14204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:41.257166 14204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:41.257411 14204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:41.275978 14204 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:44993
I20260808 11:49:41.275996 14297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:44993 every 8 connection(s)
I20260808 11:49:41.277328 14204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:41.281000 14302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:41.285439 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14204
I20260808 11:49:41.285601 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:41.286228 13945 external_mini_cluster.cc:1428] Setting key d33bfc7a25a051c9005f3b790d282335
I20260808 11:49:41.288146 14302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:41 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189781, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:49:41.310989 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.291141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44993 (local address 127.0.0.1:42480)
0808 11:49:41.292451 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.292511 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:41.294319 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:41.295250 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:41.295255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:41.295791 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:41.296108 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:41.296128 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.298411 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.298414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:41.299585 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:41.299601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.300937 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.300942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:41.301058 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:41.301713 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:41.301734 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:41.303862 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:41.306899 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:41.306911 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:41.306921 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:41.307286 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:41.307699 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:41.307706 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:41.307712 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:41.307899 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:41.308607 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:41.308616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:41.308842 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 11:49:41.309066 (+ 224us) client_negotiation.cc:240] Negotiation successful
0808 11:49:41.309658 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"mutex_wait_us":153,"spinlock_wait_cycles":43776,"thread_start_us":838,"threads_started":1}
I20260808 11:49:41.311026 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.291141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.13.158.126:44993)
0808 11:49:41.291921 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:41.291936 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:41.292660 (+ 724us) server_negotiation.cc:407] Connection header received
0808 11:49:41.294682 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:41.294691 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:41.294997 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:41.295218 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:41.296336 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.298255 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.299751 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.300697 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.300902 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:41.304072 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:41.304079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:41.304089 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:41.304114 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:41.306783 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:41.307437 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:41.307449 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:41.307451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:41.307587 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:41.308016 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:41.308026 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:41.308029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:41.308472 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:41.308682 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:41.309200 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 11:49:41.309493 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":318,"threads_started":1}
I20260808 11:49:41.314107 14302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:49:41.315375 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.314969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55602)
0808 11:49:41.315140 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.315267 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 11:49:41.315655 14234 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
I20260808 11:49:41.317238 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.316893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55612)
0808 11:49:41.317046 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.317110 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 11:49:41.317440 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:46499: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.158.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab.
Entry for principal kudu/127.13.158.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab.
I20260808 11:49:41.357905 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.125:46499
--webserver_interface=127.13.158.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab
--principal=kudu/127.13.158.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:44993,127.13.158.125:46499,127.13.158.124:35685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:49:41.366776 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } attempt: 1
I20260808 11:49:41.368075 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.367523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55624)
0808 11:49:41.367794 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.367913 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260808 11:49:41.369200 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.368845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55630)
0808 11:49:41.369011 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.369078 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 11:49:41.369458 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:46499: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
I20260808 11:49:41.437767 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } attempt: 2
I20260808 11:49:41.439056 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.438497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55644)
0808 11:49:41.438823 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.438921 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 11:49:41.440171 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.439829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55650)
0808 11:49:41.439984 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.440053 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 11:49:41.440469 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:46499: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
I20260808 11:49:41.531761 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } attempt: 3
I20260808 11:49:41.532987 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.532546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55660)
0808 11:49:41.532763 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.532856 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 11:49:41.534066 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.533749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55670)
0808 11:49:41.533880 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.533945 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 11:49:41.534286 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:46499: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
I20260808 11:49:41.677569 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } attempt: 4
I20260808 11:49:41.678987 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.678361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55682)
0808 11:49:41.678693 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.678843 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260808 11:49:41.680018 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.679696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55692)
0808 11:49:41.679828 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.679909 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 11:49:41.680259 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:46499: Network error: Client connection negotiation failed: client connection to 127.13.158.125:46499: connect: Connection refused (error 111)
W20260808 11:49:41.697960 14310 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:49:41.698285 14310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:41.698396 14310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:41.698516 14310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:41.705526 14310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:41.705650 14310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:41.705754 14310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:41.705853 14310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:41.705937 14310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:41.715770 14310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:44993,127.13.158.125:46499,127.13.158.124:35685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab
--principal=kudu/127.13.158.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.125:46499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.158.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:41.717144 14310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:41.718501 14310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:41.730453 14330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:41.730520 14321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:41.730494 14326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:41.731585 14310 server_base.cc:1034] running on GCE node
I20260808 11:49:41.732446 14310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:41.733814 14310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:41.735051 14310 hybrid_clock.cc:648] HybridClock initialized: now 1786189781734986 us; error 43 us; skew 500 ppm
I20260808 11:49:41.735373 14310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:41 dist-test-slave-39l8 krb5kdc[14188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189781, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:41.738523 14310 init.cc:377] Logged in from keytab as kudu/127.13.158.125@KRBTEST.COM (short username kudu)
I20260808 11:49:41.739916 14310 webserver.cc:460] Webserver started at http://127.13.158.125:38011/ using document root <none> and password file <none>
I20260808 11:49:41.740578 14310 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:41.740665 14310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:41.740959 14310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:41.743940 14310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/data/instance:
uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4"
format_stamp: "Formatted at 2026-08-08 11:49:41 on dist-test-slave-39l8"
server_key: "345ccd7a89fe0ac35294244dff3e37ad"
server_key_iv: "087c318ff573c57b8bde1444d4653a5a"
server_key_version: "encryptionkey@0"
I20260808 11:49:41.744762 14310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/wal/instance:
uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4"
format_stamp: "Formatted at 2026-08-08 11:49:41 on dist-test-slave-39l8"
server_key: "345ccd7a89fe0ac35294244dff3e37ad"
server_key_iv: "087c318ff573c57b8bde1444d4653a5a"
server_key_version: "encryptionkey@0"
I20260808 11:49:41.749722 14310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 11:49:41.753095 14341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:41.754405 14310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:49:41.754618 14310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/wal
uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4"
format_stamp: "Formatted at 2026-08-08 11:49:41 on dist-test-slave-39l8"
server_key: "345ccd7a89fe0ac35294244dff3e37ad"
server_key_iv: "087c318ff573c57b8bde1444d4653a5a"
server_key_version: "encryptionkey@0"
I20260808 11:49:41.754904 14310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:41.775228 14310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:41.779335 14310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:41.779636 14310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:41.798133 14310 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.125:46499
I20260808 11:49:41.798166 14423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.125:46499 every 8 connection(s)
I20260808 11:49:41.799507 14310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
I20260808 11:49:41.800599 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14310
I20260808 11:49:41.800794 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-1/wal/instance
I20260808 11:49:41.801273 13945 external_mini_cluster.cc:1428] Setting key 1e76e750a3d420e978be0e67d5141d87
I20260808 11:49:41.803413 14424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:41.809587 14424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:41 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189781, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:49:41.832710 14429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.812066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.13.158.126:44993)
0808 11:49:41.812520 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:41.812526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:41.813635 (+ 1109us) server_negotiation.cc:407] Connection header received
0808 11:49:41.816378 (+ 2743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:41.816387 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:41.816480 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:41.816627 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:41.818076 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.819347 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.820853 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.821586 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.821745 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:41.825519 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:41.825528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:41.825536 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:41.825600 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:41.829345 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:41.830083 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:41.830092 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:41.830098 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:41.830252 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:41.830651 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:41.830659 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:41.830665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:41.831116 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:41.831287 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:41.831924 (+ 637us) server_negotiation.cc:299] Negotiation successful
0808 11:49:41.832193 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":218,"threads_started":1}
I20260808 11:49:41.833312 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.812670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44993 (local address 127.0.0.1:42490)
0808 11:49:41.813444 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.813494 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:41.816115 (+ 2621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:41.816826 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:41.816833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:41.817280 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:41.817863 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:41.817879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.819508 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.819512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:41.820686 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:41.820699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.821810 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.821815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:41.822008 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:41.822815 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:41.822836 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:41.825219 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:41.829488 (+ 4269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:41.829495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:41.829507 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:41.829944 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:41.830364 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:41.830368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:41.830370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:41.830531 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:41.831240 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:41.831247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:41.831537 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 11:49:41.831799 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 11:49:41.832168 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":289,"threads_started":1}
I20260808 11:49:41.835309 14424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.158.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.124@KRBTEST.COM" created.
Aug 08 11:49:41 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189781, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:49:41.850379 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.836146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55696)
0808 11:49:41.836324 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.836344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:41.836462 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:41.837389 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:41.837396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:41.837719 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:41.837846 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:41.837854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.839263 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.839266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:41.840421 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:41.840435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.841242 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.841248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:41.841406 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:41.841945 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:41.841960 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:41.844003 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:41.848075 (+ 4072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:41.848079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:41.848082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:41.848432 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:41.848860 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:41.848863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:41.848865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:41.848998 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:41.849724 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:41.849728 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:41.849834 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 11:49:41.849943 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 11:49:41.850128 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 11:49:41.850925 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.836338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55696 (local address 127.13.158.125:46499)
0808 11:49:41.836952 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:41.836957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:41.836981 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:49:41.837018 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:41.837023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:41.837211 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:41.837330 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:41.837988 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.839127 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.840562 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.841079 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.841227 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:41.844248 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:41.844262 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:41.844280 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:41.844327 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:41.847931 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:41.848569 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:41.848580 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:41.848586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:41.848748 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:41.849105 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:41.849112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:41.849118 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:41.849584 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:41.849838 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:41.850130 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 11:49:41.850405 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":361,"threads_started":1}
I20260808 11:49:41.853302 14424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:49:41.854565 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.854040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55798)
0808 11:49:41.854235 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.854381 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 11:49:41.854948 14343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
I20260808 11:49:41.857331 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.856925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55810)
0808 11:49:41.857093 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.857196 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 11:49:41.857554 14424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab.
Entry for principal kudu/127.13.158.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab.
I20260808 11:49:41.878082 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.124:35685
--webserver_interface=127.13.158.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab
--principal=kudu/127.13.158.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:44993,127.13.158.125:46499,127.13.158.124:35685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:49:41.906930 14424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 1
I20260808 11:49:41.908311 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.907674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55816)
0808 11:49:41.907942 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.908087 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 11:49:41.909519 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.909034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55832)
0808 11:49:41.909245 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.909358 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 11:49:41.909754 14424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
I20260808 11:49:41.978003 14424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 2
I20260808 11:49:41.979271 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.978693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55848)
0808 11:49:41.978973 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.979085 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 11:49:41.979543 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } attempt: 5
I20260808 11:49:41.980540 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.980028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55850)
0808 11:49:41.980267 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.980367 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260808 11:49:41.980820 14424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Aug 08 11:49:41 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189781, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:49:41.993114 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.980219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55704 (local address 127.13.158.125:46499)
0808 11:49:41.980420 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:41.980428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:41.980484 (+ 56us) server_negotiation.cc:407] Connection header received
0808 11:49:41.980769 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:41.980777 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:41.980899 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:41.981001 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:41.981799 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.982488 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.983713 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.984163 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.984308 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:41.987967 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:41.987972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:41.987975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:41.988006 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:41.990550 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:41.991353 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:41.991357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:41.991359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:41.991469 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:41.991872 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:41.991875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:41.991877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:41.992161 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:41.992272 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:41.992713 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 11:49:41.992877 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 11:49:41.993288 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.980157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55704)
0808 11:49:41.980406 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.980425 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:41.980572 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:41.981038 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:41.981042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:41.981436 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:41.981610 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:41.981620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.982629 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.982637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:41.983553 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:41.983566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:41.984334 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:41.984340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:41.984564 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:41.985367 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:41.985384 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:41.987783 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:41.990697 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:41.990725 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:41.990729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:41.991241 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:41.991581 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:41.991585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:41.991589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:41.991761 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:41.992287 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:41.992292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:41.992466 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 11:49:41.992625 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 11:49:41.992925 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260808 11:49:41.994415 14302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:49:41.995590 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.995137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55856)
0808 11:49:41.995328 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.995422 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 11:49:41.996600 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:41.996184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55860)
0808 11:49:41.996384 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:41.996482 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 11:49:41.996843 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
I20260808 11:49:42.048146 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 1
I20260808 11:49:42.049660 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.048938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55872)
0808 11:49:42.049242 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.049380 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 11:49:42.050899 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.050370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55882)
0808 11:49:42.050576 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.050734 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 11:49:42.051204 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
I20260808 11:49:42.072093 14424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 3
I20260808 11:49:42.073254 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.072809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55886)
0808 11:49:42.072999 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.073089 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 11:49:42.074440 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.074000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55892)
0808 11:49:42.074217 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.074312 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260808 11:49:42.074663 14424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
I20260808 11:49:42.119480 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 2
I20260808 11:49:42.120854 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.120228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55898)
0808 11:49:42.120491 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.120632 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 11:49:42.121975 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.121548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55908)
0808 11:49:42.121730 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.121825 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 11:49:42.122195 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
I20260808 11:49:42.217918 14424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 4
I20260808 11:49:42.219084 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.218571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55914)
0808 11:49:42.218843 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.218927 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 11:49:42.220194 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.219813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55922)
0808 11:49:42.219966 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.220076 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:49:42.220542 14424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
W20260808 11:49:42.224161 14440 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:49:42.224517 14440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:42.224622 14440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:42.224750 14440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 11:49:42.228423 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 3
I20260808 11:49:42.229502 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.228995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55926)
0808 11:49:42.229199 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.229297 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 11:49:42.230580 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.230174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55934)
0808 11:49:42.230364 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.230444 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260808 11:49:42.230810 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:35685: Network error: Client connection negotiation failed: client connection to 127.13.158.124:35685: connect: Connection refused (error 111)
W20260808 11:49:42.232903 14440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:42.232980 14440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:42.233103 14440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:42.233193 14440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:42.233321 14440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:42.242946 14440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:44993,127.13.158.125:46499,127.13.158.124:35685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab
--principal=kudu/127.13.158.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.124:35685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.158.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:42.244294 14440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:42.245709 14440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:42.257543 14450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:42.257656 14454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:42.257622 14451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:42.258736 14440 server_base.cc:1034] running on GCE node
I20260808 11:49:42.259636 14440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:42.261096 14440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:42.262377 14440 hybrid_clock.cc:648] HybridClock initialized: now 1786189782262315 us; error 64 us; skew 500 ppm
I20260808 11:49:42.262777 14440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189782, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:42.265956 14440 init.cc:377] Logged in from keytab as kudu/127.13.158.124@KRBTEST.COM (short username kudu)
I20260808 11:49:42.267630 14440 webserver.cc:460] Webserver started at http://127.13.158.124:46515/ using document root <none> and password file <none>
I20260808 11:49:42.268303 14440 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:42.268392 14440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:42.268662 14440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:42.271659 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/data/instance:
uuid: "69bd03614f23479eb50efa35d57388ab"
format_stamp: "Formatted at 2026-08-08 11:49:42 on dist-test-slave-39l8"
server_key: "ecb0715d0471e447bb6fcb4aec7dba96"
server_key_iv: "4be63cdbacaa593315b246407944926e"
server_key_version: "encryptionkey@0"
I20260808 11:49:42.272464 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/wal/instance:
uuid: "69bd03614f23479eb50efa35d57388ab"
format_stamp: "Formatted at 2026-08-08 11:49:42 on dist-test-slave-39l8"
server_key: "ecb0715d0471e447bb6fcb4aec7dba96"
server_key_iv: "4be63cdbacaa593315b246407944926e"
server_key_version: "encryptionkey@0"
I20260808 11:49:42.277786 14440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:49:42.281354 14469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:42.282684 14440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:49:42.282925 14440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/wal
uuid: "69bd03614f23479eb50efa35d57388ab"
format_stamp: "Formatted at 2026-08-08 11:49:42 on dist-test-slave-39l8"
server_key: "ecb0715d0471e447bb6fcb4aec7dba96"
server_key_iv: "4be63cdbacaa593315b246407944926e"
server_key_version: "encryptionkey@0"
I20260808 11:49:42.283141 14440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:42.294389 14440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:42.298771 14440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:42.299074 14440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:42.317349 14440 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.124:35685
I20260808 11:49:42.317370 14536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.124:35685 every 8 connection(s)
I20260808 11:49:42.318693 14440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
I20260808 11:49:42.320215 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14440
I20260808 11:49:42.320410 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189414476033-13945-0/minicluster-data/master-2/wal/instance
I20260808 11:49:42.321007 13945 external_mini_cluster.cc:1428] Setting key c69a5b772e5bce6d9145e160c65790bc
I20260808 11:49:42.323300 14537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:42.331676 14537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189782, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:49:42.345423 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.322935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55938 (local address 127.13.158.124:35685)
0808 11:49:42.323794 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.323805 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.323846 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:49:42.325394 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.325415 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.325801 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.326143 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.328066 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.329365 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.331229 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.331863 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.332011 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.337310 (+ 5299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.337321 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.337335 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.337379 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.340914 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.341758 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.341764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.341769 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.341938 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.342329 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.342332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.342334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.342848 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.343169 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.343735 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.344131 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":418,"threads_started":1}
I20260808 11:49:42.350346 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.334447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.13.158.126:44993)
0808 11:49:42.335026 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.335033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.335506 (+ 473us) server_negotiation.cc:407] Connection header received
0808 11:49:42.335699 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.335704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.335778 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.335864 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.337357 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.338398 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.340060 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.340729 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.340858 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.344251 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.344257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.344262 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.344300 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.347343 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.348163 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.348167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.348170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.348316 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.348807 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.348823 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.348824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.349245 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.349428 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.349931 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.350084 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":302,"threads_started":1}
I20260808 11:49:42.350559 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.334533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:44993 (local address 127.0.0.1:42496)
0808 11:49:42.335320 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.335387 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:42.335560 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:42.335965 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:42.335972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:42.336817 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:42.337198 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.337211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.338574 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.338579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.339900 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.339913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.340912 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.340916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.341109 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.341742 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:42.341765 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:42.344048 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:42.347475 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.347483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.347489 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.348023 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.348482 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.348487 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.348490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.348652 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.349403 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:42.349409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:42.349561 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 11:49:42.349778 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 11:49:42.350097 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":445,"threads_started":1}
I20260808 11:49:42.351897 14537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189782, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:49:42.364956 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.348663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.13.158.126:44993)
0808 11:49:42.349427 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.349433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.349454 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:49:42.349500 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.349503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.349553 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.349629 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.351485 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.352395 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.354162 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.354569 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.354700 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.358919 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.358923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.358926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.358954 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.361936 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.362788 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.362791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.362793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.362909 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.363320 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.363324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.363327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.363697 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.363985 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.364301 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.364533 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":342,"threads_started":1}
I20260808 11:49:42.365666 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.352605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:46499 (local address 127.0.0.1:55716)
0808 11:49:42.352774 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.352793 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:42.352886 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:42.353172 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:42.353177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:42.353598 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:42.353773 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.353783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.354695 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.354766 (+ 71us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.355773 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.355785 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.356484 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.356490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.356672 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.357207 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:42.357229 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:42.359765 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:42.363346 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.363350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.363353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.363810 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.364222 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.364227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.364230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.364395 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.364963 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:42.364970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:42.365135 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 11:49:42.365269 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 11:49:42.365475 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 11:49:42.366000 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.352733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.13.158.125:46499)
0808 11:49:42.352901 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.352906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.352922 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:49:42.352973 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.352976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.353031 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.353110 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.353921 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.354557 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.355933 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.356332 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.356464 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.359931 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.359939 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.359945 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.359991 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.363220 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.363920 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.363927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.363930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.364088 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.364525 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.364530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.364533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.364817 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.364940 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.365421 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.365635 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:49:42.367072 14537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189782, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:49:42.380337 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.367986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55948)
0808 11:49:42.368204 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.368219 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:42.368307 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:42.368589 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:42.368592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:42.368907 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:42.369027 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.369034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.370036 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.370042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.371033 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.371045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.371815 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.371821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.372090 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.372793 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:42.372809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:42.374830 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:42.377944 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.377951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.377954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.378299 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.378669 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.378674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.378677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.378853 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.379325 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:42.379330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:42.379467 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 11:49:42.379606 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 11:49:42.379929 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 11:49:42.380479 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.367950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55948 (local address 127.13.158.124:35685)
0808 11:49:42.368129 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.368135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.368298 (+ 163us) server_negotiation.cc:407] Connection header received
0808 11:49:42.368383 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.368388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.368461 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.368566 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.369169 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.369901 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.371176 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.371643 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.371747 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.374995 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.374999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.375002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.375029 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.377811 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.378415 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.378422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.378425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.378551 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.378958 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.378964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.378967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.379210 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.379359 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.379707 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.380091 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":58}
I20260808 11:49:42.381529 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.367017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.13.158.125:46499)
0808 11:49:42.367168 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.367174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.367485 (+ 311us) server_negotiation.cc:407] Connection header received
0808 11:49:42.367658 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.367662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.367741 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.368039 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.369274 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.369931 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.371262 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.371819 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.371954 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.376226 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.376231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.376234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.376263 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.378594 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.379260 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.379268 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.379272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.379412 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.379816 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.379821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.379830 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.380070 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.380348 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.381127 (+ 779us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.381280 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 11:49:42.386168 14537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: Bootstrap starting.
I20260808 11:49:42.386581 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:42.390221 14537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:42.391363 14537 log.cc:824] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:42.390012 14270 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7f2a20c9149d48c8b121739620b9ce4f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42506
I20260808 11:49:42.394014 14537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: No bootstrap required, opened a new log
I20260808 11:49:42.400862 14537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } }
I20260808 11:49:42.401204 14537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69bd03614f23479eb50efa35d57388ab, State: Initialized, Role: FOLLOWER
I20260808 11:49:42.401619 14537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } }
I20260808 11:49:42.402524 14547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.402936 14547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:42.403447 14537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:42.407954 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 4
W20260808 11:49:42.414814 14563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:49:42.414935 14563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189781, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
I20260808 11:49:42.410681 14440 master_runner.cc:186] Error getting master registration for 127.13.158.126:44993: OK, instance_id { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" instance_seqno: 1786189781270599 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:49:42.422798 14440 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 11:49:42.425333 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.409269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.13.158.124:35685)
0808 11:49:42.409433 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.409437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.409451 (+ 14us) server_negotiation.cc:407] Connection header received
0808 11:49:42.409493 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.409496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.409545 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.409665 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.410669 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.411871 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.413731 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.414523 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.414644 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.418599 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.418610 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.418619 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.418646 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.422473 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.423265 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.423271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.423274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.423402 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.423832 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.423837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.423840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.424213 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.424391 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.424782 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.424977 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 11:49:42.425328 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.408564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55954)
0808 11:49:42.408797 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.408816 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:42.408932 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:42.409702 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:42.409710 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:42.410265 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:42.410482 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.410496 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.412040 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.412050 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.413533 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.413550 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.414867 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.414872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.415072 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.415761 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:42.415783 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:42.418427 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:42.422615 (+ 4188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.422620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.422623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.423148 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.423534 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.423538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.423542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.423709 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.424346 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:42.424352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:42.424522 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 11:49:42.424666 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 11:49:42.424948 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 11:49:42.431516 14302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f: Bootstrap starting.
I20260808 11:49:42.435652 14302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:42.436909 14302 log.cc:824] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:42.439759 14302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f: No bootstrap required, opened a new log
I20260808 11:49:42.445508 14302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } }
I20260808 11:49:42.445897 14302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f2a20c9149d48c8b121739620b9ce4f, State: Initialized, Role: FOLLOWER
I20260808 11:49:42.446348 14302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } }
I20260808 11:49:42.447292 14566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.447757 14566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:42.448491 14302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:42.448591 14270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 11:49:42.448798 14270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:42.452442 14270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } }
I20260808 11:49:42.454934 14502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f2a20c9149d48c8b121739620b9ce4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "69bd03614f23479eb50efa35d57388ab"
I20260808 11:49:42.455427 14502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:42.459107 14502 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f2a20c9149d48c8b121739620b9ce4f in term 1.
I20260808 11:49:42.459839 14233 leader_election.cc:302] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69bd03614f23479eb50efa35d57388ab, 7f2a20c9149d48c8b121739620b9ce4f; no voters:
I20260808 11:49:42.460363 14270 leader_election.cc:288] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [CANDIDATE]: Term 1 election: Requested vote from peers 451f4aaf0c4f4b339eb9cd2b85f5dae4 (127.13.158.125:46499), 69bd03614f23479eb50efa35d57388ab (127.13.158.124:35685)
I20260808 11:49:42.460363 14566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:42.474474 14566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [term 1 LEADER]: Becoming Leader. State: Replica: 7f2a20c9149d48c8b121739620b9ce4f, State: Running, Role: LEADER
I20260808 11:49:42.475235 14566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } }
I20260808 11:49:42.481139 14580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2a20c9149d48c8b121739620b9ce4f. Latest consensus state: current_term: 1 leader_uuid: "7f2a20c9149d48c8b121739620b9ce4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.481717 14580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:42.487633 14592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:42.506656 14592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 11:49:42.510262 14238 tablet.cc:2369] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:49:42.499531 14204 master_runner.cc:186] Error getting master registration for 127.13.158.125:46499: OK, instance_id { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" instance_seqno: 1786189781792836 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:49:42.516844 14204 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 11:49:42.521294 14424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } attempt: 5
I20260808 11:49:42.528331 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.496661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.13.158.125:46499)
0808 11:49:42.497484 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.497490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.498927 (+ 1437us) server_negotiation.cc:407] Connection header received
0808 11:49:42.499191 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.499198 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.499287 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.499362 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.513770 (+ 14408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.514908 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.517052 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.517692 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.517790 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.521034 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.521043 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.521050 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.521112 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.524945 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.525825 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.525831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.525837 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.525998 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.526477 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.526482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.526486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.526880 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.527132 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.527716 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.527949 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691}
I20260808 11:49:42.528393 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.496658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.13.158.126:44993)
0808 11:49:42.496887 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.496895 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.500395 (+ 3500us) server_negotiation.cc:407] Connection header received
0808 11:49:42.500726 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.500744 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.500839 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.500920 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.504455 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.505559 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.508520 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.509276 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.509386 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.516022 (+ 6636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.516029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.516034 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.516069 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.521135 (+ 5066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.524928 (+ 3793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.524934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.524937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.525137 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.526291 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.526296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.526299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.526683 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.526981 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.527840 (+ 859us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.528061 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 11:49:42 dist-test-slave-39l8 krb5kdc[14188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189781, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
I20260808 11:49:42.539510 14502 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [term 1 FOLLOWER]: Refusing update from remote peer 7f2a20c9149d48c8b121739620b9ce4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:49:42.540501 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.497744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.13.158.124:35685)
0808 11:49:42.497909 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.497914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.511806 (+ 13892us) server_negotiation.cc:407] Connection header received
0808 11:49:42.523074 (+ 11268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.523083 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.523192 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.523285 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.525534 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.526542 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.528531 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.529259 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.529358 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.533212 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.533220 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.533237 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.533283 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.537056 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.537884 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.537890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.537899 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.538063 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.538530 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.538535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.538538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.538937 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.539206 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.539823 (+ 617us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.540084 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:49:42.540586 14580 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:49:42.551029 14477 tablet.cc:2369] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:49:42.561137 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.522772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:35685 (local address 127.0.0.1:55978)
0808 11:49:42.523069 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:42.523089 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:42.523198 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:42.526915 (+ 3717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:42.526923 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:42.527471 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:42.527665 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.527677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.530724 (+ 3047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.530732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.532179 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:42.532194 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.533340 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.533348 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:42.533576 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.534374 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:42.534397 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:42.537105 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:42.545466 (+ 8361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.545471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.545475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.546103 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.546721 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:42.546729 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:42.546735 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:42.546992 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:42.559579 (+ 12587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:42.559588 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:42.559851 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 11:49:42.560030 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 11:49:42.560487 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260808 11:49:42.562923 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:42.522334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55978 (local address 127.13.158.124:35685)
0808 11:49:42.523570 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.523575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.523596 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:49:42.523647 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.523651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.523724 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.527819 (+ 4095us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.528043 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.529484 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.532362 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:42.533142 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:42.533910 (+ 768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:42.541486 (+ 7576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:42.541493 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:42.541497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:42.541534 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:42.545294 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.546306 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.546314 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.546320 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.546526 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:42.558844 (+ 12318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:42.558852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:42.558858 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:42.559364 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:42.561574 (+ 2210us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:42.561754 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 11:49:42.562127 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1112,"thread_start_us":950,"threads_started":1}
I20260808 11:49:42.569044 14424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4: Bootstrap starting.
I20260808 11:49:42.570508 14547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2a20c9149d48c8b121739620b9ce4f. Latest consensus state: current_term: 1 leader_uuid: "7f2a20c9149d48c8b121739620b9ce4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.570871 14547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:42.572055 14600 mvcc.cc:205] Tried to move back new op lower bound from 7316233349270503424 to 7316233349025718272. Current Snapshot: MvccSnapshot[applied={T|T < 7316233349270503424}]
I20260808 11:49:42.578224 14424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:42.582378 14424 log.cc:824] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:42.587281 14547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f2a20c9149d48c8b121739620b9ce4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.587576 14547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:42.591856 14580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f2a20c9149d48c8b121739620b9ce4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.592182 14580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:42.592751 14592 catalog_manager.cc:1347] Generated new cluster ID: 61577d86d742491599bbd8a2090f9757
I20260808 11:49:42.592868 14592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:42.594367 14424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4: No bootstrap required, opened a new log
I20260808 11:49:42.612818 14424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } }
I20260808 11:49:42.613507 14424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451f4aaf0c4f4b339eb9cd2b85f5dae4, State: Initialized, Role: FOLLOWER
I20260808 11:49:42.614302 14424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } }
I20260808 11:49:42.621045 14592 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:42.622958 14592 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:42.631315 14607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.631755 14607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:42.632776 14424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:42.633209 14376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:42.637058 14376 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [term 1 FOLLOWER]: Refusing update from remote peer 7f2a20c9149d48c8b121739620b9ce4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:49:42.638008 14566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:49:42.642253 14378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f2a20c9149d48c8b121739620b9ce4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4"
I20260808 11:49:42.654969 14592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f: Generated new TSK 0
I20260808 11:49:42.656316 14592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:42.668001 14610 mvcc.cc:205] Tried to move back new op lower bound from 7316233349270503424 to 7316233349025718272. Current Snapshot: MvccSnapshot[applied={T|T < 7316233349270503424}]
I20260808 11:49:42.672971 14607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2a20c9149d48c8b121739620b9ce4f. Latest consensus state: current_term: 1 leader_uuid: "7f2a20c9149d48c8b121739620b9ce4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.673396 14607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:42.673907 14378 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f2a20c9149d48c8b121739620b9ce4f for term 1 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260808 11:49:42.676107 14616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f2a20c9149d48c8b121739620b9ce4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.676630 14616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:42.676461 14580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f2a20c9149d48c8b121739620b9ce4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f2a20c9149d48c8b121739620b9ce4f" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 44993 } } peers { permanent_uuid: "451f4aaf0c4f4b339eb9cd2b85f5dae4" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 46499 } } peers { permanent_uuid: "69bd03614f23479eb50efa35d57388ab" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 35685 } } }
I20260808 11:49:42.676781 14580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2a20c9149d48c8b121739620b9ce4f [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:42.732302 14632 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260808 11:49:42.796648 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.777780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.13.158.125:46499)
0808 11:49:42.778267 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.778273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.792255 (+ 13982us) server_negotiation.cc:407] Connection header received
0808 11:49:42.793620 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.793630 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.793745 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.793828 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.796373 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55726: BlockingRecv error: recv got EOF from 127.0.0.1:55726 (error 108)
Metrics: {"server-negotiator.queue_time_us":295}
W20260808 11:49:42.799082 14601 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.777981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.13.158.124:35685)
0808 11:49:42.778236 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.778246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.795172 (+ 16926us) server_negotiation.cc:407] Connection header received
0808 11:49:42.797235 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.797247 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.797345 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.797428 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.798824 (+ 1396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55984: BlockingRecv error: recv got EOF from 127.0.0.1:55984 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 11:49:42.802368 14545 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.784056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.13.158.126:44993)
0808 11:49:42.784255 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.784262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.800106 (+ 15844us) server_negotiation.cc:407] Connection header received
0808 11:49:42.800764 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.800771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.800885 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.800976 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.802075 (+ 1099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42516: BlockingRecv error: recv got EOF from 127.0.0.1:42516 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 11:49:42.836293 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.833815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.13.158.125:46499)
0808 11:49:42.834027 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.834034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.834566 (+ 532us) server_negotiation.cc:407] Connection header received
0808 11:49:42.834857 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.834865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.835009 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.835124 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.836038 (+ 914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55742: BlockingRecv error: recv got EOF from 127.0.0.1:55742 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 11:49:42.844697 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.840994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.13.158.125:46499)
0808 11:49:42.841153 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.841159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.843202 (+ 2043us) server_negotiation.cc:407] Connection header received
0808 11:49:42.843416 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.843422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.843522 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.843626 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.844498 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55748: BlockingRecv error: recv got EOF from 127.0.0.1:55748 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 11:49:42.856746 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.850060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55752 (local address 127.13.158.125:46499)
0808 11:49:42.850219 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.850225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.855257 (+ 5032us) server_negotiation.cc:407] Connection header received
0808 11:49:42.855453 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.855458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.855568 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.855675 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.856515 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55752: BlockingRecv error: recv got EOF from 127.0.0.1:55752 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 11:49:42.865792 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.863633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55756 (local address 127.13.158.125:46499)
0808 11:49:42.863846 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.863852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.864369 (+ 517us) server_negotiation.cc:407] Connection header received
0808 11:49:42.864578 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.864583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.864681 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.864784 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.865596 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55756: BlockingRecv error: recv got EOF from 127.0.0.1:55756 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
W20260808 11:49:42.879637 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.877362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.13.158.125:46499)
0808 11:49:42.877557 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.877563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.878177 (+ 614us) server_negotiation.cc:407] Connection header received
0808 11:49:42.878328 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.878332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.878422 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.878533 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.879416 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55760: BlockingRecv error: recv got EOF from 127.0.0.1:55760 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 11:49:42.903636 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.901014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.13.158.125:46499)
0808 11:49:42.901228 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.901234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.901804 (+ 570us) server_negotiation.cc:407] Connection header received
0808 11:49:42.902040 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.902045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.902150 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.902255 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.903385 (+ 1130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55776: BlockingRecv error: recv got EOF from 127.0.0.1:55776 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 11:49:42.942181 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:42.939595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.13.158.125:46499)
0808 11:49:42.939831 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:42.939838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:42.940484 (+ 646us) server_negotiation.cc:407] Connection header received
0808 11:49:42.940708 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:42.940714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:42.940823 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:42.940956 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:42.941888 (+ 932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55784: BlockingRecv error: recv got EOF from 127.0.0.1:55784 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260808 11:49:43.012109 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:43.009929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55790 (local address 127.13.158.125:46499)
0808 11:49:43.010157 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:43.010163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:43.010520 (+ 357us) server_negotiation.cc:407] Connection header received
0808 11:49:43.010695 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:43.010728 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:43.010838 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:43.010967 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:43.011858 (+ 891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55790: BlockingRecv error: recv got EOF from 127.0.0.1:55790 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 11:49:43.146514 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:43.144519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.13.158.125:46499)
0808 11:49:43.144757 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:43.144765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:43.145097 (+ 332us) server_negotiation.cc:407] Connection header received
0808 11:49:43.145328 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:43.145334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:43.145414 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:43.145523 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:43.146274 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55792: BlockingRecv error: recv got EOF from 127.0.0.1:55792 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 11:49:43.407949 14435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:43.406230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.13.158.125:46499)
0808 11:49:43.406474 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:43.406481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:43.406597 (+ 116us) server_negotiation.cc:407] Connection header received
0808 11:49:43.406846 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:43.406850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:43.406924 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:43.407008 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:43.407717 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55796: BlockingRecv error: recv got EOF from 127.0.0.1:55796 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 11:49:43.416821 14563 catalog_manager.cc:1259] Loaded cluster ID: 61577d86d742491599bbd8a2090f9757
I20260808 11:49:43.416913 14563 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: loading cluster ID for follower catalog manager: success
I20260808 11:49:43.419361 14563 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: acquiring CA information for follower catalog manager: success
I20260808 11:49:43.421126 14563 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 69bd03614f23479eb50efa35d57388ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:49:43.736699 14632 catalog_manager.cc:1259] Loaded cluster ID: 61577d86d742491599bbd8a2090f9757
I20260808 11:49:43.736838 14632 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4: loading cluster ID for follower catalog manager: success
I20260808 11:49:43.739504 14632 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4: acquiring CA information for follower catalog manager: success
I20260808 11:49:43.741307 14632 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 451f4aaf0c4f4b339eb9cd2b85f5dae4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:49:43.935669 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:43.924427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.13.158.125:46499)
0808 11:49:43.929923 (+ 5496us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:43.929927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:43.929943 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:49:43.930010 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:43.930014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:43.930143 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:43.930309 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:43.930831 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.931700 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.933128 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.933652 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.933803 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:43.934319 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:43.934976 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:43.935140 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:43.935245 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 11:49:43.935324 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5344,"thread_start_us":5229,"threads_started":1}
I20260808 11:49:43.953070 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:43.942782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.13.158.124:35685)
0808 11:49:43.947058 (+ 4276us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:43.947063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:43.947079 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:49:43.947178 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:43.947182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:43.947328 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:43.947467 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:43.947966 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.948664 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.950625 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.951075 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.951178 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:43.951746 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:43.952377 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:43.952486 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:43.952654 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 11:49:43.952731 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4173,"thread_start_us":4072,"threads_started":1}
I20260808 11:49:43.980402 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:43.961033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55824 (local address 127.13.158.125:46499)
0808 11:49:43.961273 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:43.961282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:43.972106 (+ 10824us) server_negotiation.cc:407] Connection header received
0808 11:49:43.972612 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:43.972621 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:43.972720 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:43.972815 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:43.973579 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.974781 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.977613 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.978282 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.978391 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:43.979439 (+ 1048us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:43.979577 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:43.979647 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:43.979941 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 11:49:43.980044 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 11:49:43.982410 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:43.961180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.13.158.124:35685)
0808 11:49:43.961364 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:43.961370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:43.973484 (+ 12114us) server_negotiation.cc:407] Connection header received
0808 11:49:43.973959 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:43.973967 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:43.974088 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:43.974171 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:43.975016 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.976238 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.979062 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.979973 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.980096 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:43.981058 (+ 962us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:43.981302 (+ 244us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:43.981395 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:43.981850 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 11:49:43.981964 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:49:43.989050 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:43.960610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.13.158.126:44993)
0808 11:49:43.983829 (+ 23219us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:43.983834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:43.983849 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:49:43.983941 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:43.983944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:43.984224 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:43.984419 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:43.984849 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.985547 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.987000 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:43.987381 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:43.987640 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:43.988093 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:43.988401 (+ 308us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:43.988531 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:43.988671 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 11:49:43.988751 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23106,"thread_start_us":23007,"threads_started":1}
I20260808 11:49:43.994694 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14204
I20260808 11:49:43.996007 14293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:44.140570 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14204
I20260808 11:49:44.172269 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14310
I20260808 11:49:44.173437 14416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:44.277633 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14310
I20260808 11:49:44.307082 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14440
I20260808 11:49:44.308295 14529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:44.418378 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14440
2026-08-08T11:49:44Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5205 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:49:44 dist-test-slave-39l8 krb5kdc[14757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:49:44 dist-test-slave-39l8 krb5kdc[14757](info): set up 2 sockets
Aug 08 11:49:44 dist-test-slave-39l8 krb5kdc[14757](info): commencing operation
krb5kdc: starting...
W20260808 11:49:45.817462 13945 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/test-user.keytab.
Aug 08 11:49:46 dist-test-slave-39l8 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:49:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:49:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
Entry for principal kudu/127.13.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab.
I20260808 11:49:46.149284 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:32979
--webserver_interface=127.13.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:32979,127.13.158.125:40219,127.13.158.124:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:49:46.485261 14774 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:49:46.485643 14774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:46.485767 14774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:46.485900 14774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:46.493033 14774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:46.493180 14774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:46.493309 14774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:46.493396 14774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:46.493485 14774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:46.503314 14774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:32979,127.13.158.125:40219,127.13.158.124:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.126.keytab
--principal=kudu/127.13.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.126:32979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:46.504639 14774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:46.506016 14774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:46.518043 14783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:46.518083 14788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:46.518764 14785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:46.518761 14774 server_base.cc:1034] running on GCE node
I20260808 11:49:46.519658 14774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:46.520967 14774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:46.522198 14774 hybrid_clock.cc:648] HybridClock initialized: now 1786189786522131 us; error 31 us; skew 500 ppm
I20260808 11:49:46.522529 14774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:46 dist-test-slave-39l8 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:46.531443 14774 init.cc:377] Logged in from keytab as kudu/127.13.158.126@KRBTEST.COM (short username kudu)
I20260808 11:49:46.532894 14774 webserver.cc:460] Webserver started at http://127.13.158.126:42191/ using document root <none> and password file <none>
I20260808 11:49:46.533630 14774 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:46.533762 14774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:46.534096 14774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:46.537107 14774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/data/instance:
uuid: "0663431b6194438b856e868b4ac3a260"
format_stamp: "Formatted at 2026-08-08 11:49:46 on dist-test-slave-39l8"
server_key: "a0bbcd91b2e11b4d2a37cfbfad3be95b"
server_key_iv: "f8a87b451a2448ba842beb44dfe75232"
server_key_version: "encryptionkey@0"
I20260808 11:49:46.537916 14774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/wal/instance:
uuid: "0663431b6194438b856e868b4ac3a260"
format_stamp: "Formatted at 2026-08-08 11:49:46 on dist-test-slave-39l8"
server_key: "a0bbcd91b2e11b4d2a37cfbfad3be95b"
server_key_iv: "f8a87b451a2448ba842beb44dfe75232"
server_key_version: "encryptionkey@0"
I20260808 11:49:46.543167 14774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260808 11:49:46.546932 14798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:46.548323 14774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:49:46.548615 14774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/wal
uuid: "0663431b6194438b856e868b4ac3a260"
format_stamp: "Formatted at 2026-08-08 11:49:46 on dist-test-slave-39l8"
server_key: "a0bbcd91b2e11b4d2a37cfbfad3be95b"
server_key_iv: "f8a87b451a2448ba842beb44dfe75232"
server_key_version: "encryptionkey@0"
I20260808 11:49:46.548846 14774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:46.561982 14774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:46.566320 14774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:46.566596 14774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:46.584959 14774 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.126:32979
I20260808 11:49:46.584981 14877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.126:32979 every 8 connection(s)
I20260808 11:49:46.586299 14774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/data/info.pb
I20260808 11:49:46.589953 14878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:46.590547 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14774
I20260808 11:49:46.590775 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-0/wal/instance
I20260808 11:49:46.591236 13945 external_mini_cluster.cc:1428] Setting key 8a91e7bb98cb3167001de5958711c371
I20260808 11:49:46.597563 14878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:46 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.158.125@KRBTEST.COM; defaulting to no policy
I20260808 11:49:46.628192 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.601246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:32979 (local address 127.0.0.1:49062)
0808 11:49:46.607659 (+ 6413us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.607717 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:46.610565 (+ 2848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:46.611481 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:46.611493 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:46.612390 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:46.612913 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:46.612941 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:46.614466 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:46.614475 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:46.616471 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:46.616490 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:46.617397 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:46.617407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:46.617616 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:46.618343 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:46.618380 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:46.620957 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:46.624193 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:46.624203 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:46.624215 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:46.624562 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:46.624930 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:46.624935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:46.624939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:46.625089 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:46.625805 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:46.625817 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:46.626017 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 11:49:46.626256 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 11:49:46.626880 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6120,"mutex_wait_us":180,"thread_start_us":5901,"threads_started":1}
I20260808 11:49:46.628176 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.601246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.13.158.126:32979)
0808 11:49:46.607294 (+ 6048us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:46.607315 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:46.608027 (+ 712us) server_negotiation.cc:407] Connection header received
0808 11:49:46.610969 (+ 2942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:46.610975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:46.611203 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:46.611375 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:46.613135 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:46.614291 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:46.616632 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:46.617138 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:46.617258 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:46.621189 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:46.621197 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:46.621207 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:46.621234 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:46.624062 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:46.624695 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:46.624701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:46.624703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:46.624827 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:46.625201 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:46.625211 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:46.625213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:46.625673 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:46.625881 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:46.626387 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 11:49:46.626761 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":5763,"thread_start_us":5523,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.125@KRBTEST.COM" created.
I20260808 11:49:46.631150 14878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:49:46.632102 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.631728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52092)
0808 11:49:46.631876 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.631975 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:49:46.632331 14803 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
I20260808 11:49:46.633849 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.633541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52106)
0808 11:49:46.633687 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.633743 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 11:49:46.634011 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:40219: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab.
Entry for principal kudu/127.13.158.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab.
I20260808 11:49:46.666121 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.125:40219
--webserver_interface=127.13.158.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab
--principal=kudu/127.13.158.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:32979,127.13.158.125:40219,127.13.158.124:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:49:46.683328 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } attempt: 1
I20260808 11:49:46.684681 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.684058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52112)
0808 11:49:46.684314 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.684475 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 11:49:46.685776 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.685323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52128)
0808 11:49:46.685494 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.685599 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:49:46.685982 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:40219: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
I20260808 11:49:46.754241 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } attempt: 2
I20260808 11:49:46.755642 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.755038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52130)
0808 11:49:46.755327 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.755443 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 11:49:46.756749 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.756333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52134)
0808 11:49:46.756494 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.756600 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 11:49:46.757006 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:40219: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
I20260808 11:49:46.848299 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } attempt: 3
I20260808 11:49:46.849628 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.849092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52150)
0808 11:49:46.849338 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.849451 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 11:49:46.850812 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.850315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52156)
0808 11:49:46.850475 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.850570 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 11:49:46.851073 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:40219: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
I20260808 11:49:46.994333 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } attempt: 4
I20260808 11:49:46.995729 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.995111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52158)
0808 11:49:46.995432 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.995541 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 11:49:46.996835 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:46.996444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52172)
0808 11:49:46.996603 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:46.996693 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 11:49:46.997018 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.125:40219: Network error: Client connection negotiation failed: client connection to 127.13.158.125:40219: connect: Connection refused (error 111)
W20260808 11:49:47.000386 14885 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:49:47.000671 14885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:47.000746 14885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:47.000824 14885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:47.007936 14885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:47.008015 14885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:47.008071 14885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:47.008121 14885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:47.008173 14885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:47.018265 14885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:32979,127.13.158.125:40219,127.13.158.124:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.125.keytab
--principal=kudu/127.13.158.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.125:40219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.158.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:47.019541 14885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:47.020750 14885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:49:47.033007 14895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:47.033037 14894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:47.033216 14897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:47.033911 14885 server_base.cc:1034] running on GCE node
I20260808 11:49:47.034909 14885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:47.036310 14885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:47.037561 14885 hybrid_clock.cc:648] HybridClock initialized: now 1786189787037484 us; error 70 us; skew 500 ppm
I20260808 11:49:47.037886 14885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189787, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:47.040957 14885 init.cc:377] Logged in from keytab as kudu/127.13.158.125@KRBTEST.COM (short username kudu)
I20260808 11:49:47.042315 14885 webserver.cc:460] Webserver started at http://127.13.158.125:34471/ using document root <none> and password file <none>
I20260808 11:49:47.043076 14885 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:47.043208 14885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:47.043524 14885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:47.046463 14885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/data/instance:
uuid: "1b185c12f90342fea1067f0ee8fc1d3f"
format_stamp: "Formatted at 2026-08-08 11:49:47 on dist-test-slave-39l8"
server_key: "a4fac46ffcd6e73de9b74a0d664854d6"
server_key_iv: "586d58e000ce2e174bc6b5ba2519d903"
server_key_version: "encryptionkey@0"
I20260808 11:49:47.047344 14885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/wal/instance:
uuid: "1b185c12f90342fea1067f0ee8fc1d3f"
format_stamp: "Formatted at 2026-08-08 11:49:47 on dist-test-slave-39l8"
server_key: "a4fac46ffcd6e73de9b74a0d664854d6"
server_key_iv: "586d58e000ce2e174bc6b5ba2519d903"
server_key_version: "encryptionkey@0"
I20260808 11:49:47.052402 14885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:49:47.055781 14904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:47.056963 14885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:49:47.057142 14885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/wal
uuid: "1b185c12f90342fea1067f0ee8fc1d3f"
format_stamp: "Formatted at 2026-08-08 11:49:47 on dist-test-slave-39l8"
server_key: "a4fac46ffcd6e73de9b74a0d664854d6"
server_key_iv: "586d58e000ce2e174bc6b5ba2519d903"
server_key_version: "encryptionkey@0"
I20260808 11:49:47.057382 14885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:47.074687 14885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:47.078857 14885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:47.079128 14885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:47.097880 14885 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.125:40219
I20260808 11:49:47.097900 14991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.125:40219 every 8 connection(s)
I20260808 11:49:47.099603 14885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/data/info.pb
I20260808 11:49:47.103627 14993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:47.107873 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 14885
I20260808 11:49:47.108079 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-1/wal/instance
I20260808 11:49:47.108541 13945 external_mini_cluster.cc:1428] Setting key 8ed0ee45d6fccd17c39d60274c627efc
I20260808 11:49:47.110214 14993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189787, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:49:47.130259 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.112981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49076 (local address 127.13.158.126:32979)
0808 11:49:47.113168 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.113172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.114658 (+ 1486us) server_negotiation.cc:407] Connection header received
0808 11:49:47.116563 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.116570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.116658 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.116773 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.118253 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.119011 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.120470 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.120896 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.121032 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.124487 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.124492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.124495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.124532 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.127245 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.127974 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.127987 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.127989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.128103 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.128500 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.128510 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.128513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.128819 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.128979 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.129686 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.129920 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:49:47.131306 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.113639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:32979 (local address 127.0.0.1:49076)
0808 11:49:47.114453 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.114498 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:47.116265 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:47.117000 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:47.117008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:47.117528 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:47.118082 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.118100 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.119171 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.119175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.120313 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.120326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.121145 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.121152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.121422 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.122114 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:47.122136 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:47.124277 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:47.127395 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.127403 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.127418 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.127848 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.128214 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.128219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.128222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.128386 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.128962 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:47.128969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:47.129314 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 11:49:47.129590 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 11:49:47.130100 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":383,"threads_started":1}
I20260808 11:49:47.133014 14993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189787, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.158.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.158.124@KRBTEST.COM" created.
I20260808 11:49:47.147130 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.133694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52180)
0808 11:49:47.133897 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.133911 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:47.133988 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:47.135139 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:47.135144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:47.135553 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:47.135674 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.135682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.137005 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.137010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.137785 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.137795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.138534 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.138539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.138756 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.139367 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:47.139384 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:47.141334 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:47.144679 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.144684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.144688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.145035 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.145395 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.145402 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.145407 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.145554 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.146112 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:47.146116 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:47.146237 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 11:49:47.146382 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 11:49:47.146658 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":3}
I20260808 11:49:47.147163 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.134008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.13.158.125:40219)
0808 11:49:47.134681 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.134687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.134751 (+ 64us) server_negotiation.cc:407] Connection header received
0808 11:49:47.134808 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.134814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.134970 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.135129 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.135847 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.136871 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.137908 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.138391 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.138537 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.141534 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.141545 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.141557 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.141583 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.144545 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.145165 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.145176 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.145180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.145290 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.145653 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.145660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.145663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.145988 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.146173 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.146554 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.146792 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":537,"thread_start_us":375,"threads_started":1}
I20260808 11:49:47.150097 14993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:49:47.151225 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.150814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36128)
0808 11:49:47.150957 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.151081 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 11:49:47.151497 14909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
I20260808 11:49:47.153105 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.152752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36136)
0808 11:49:47.152917 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.152998 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 11:49:47.153343 14993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.158.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab.
Entry for principal kudu/127.13.158.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab.
I20260808 11:49:47.182225 13945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWrISxk/build/asan/bin/kudu
/tmp/dist-test-taskWrISxk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.124:32897
--webserver_interface=127.13.158.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab
--principal=kudu/127.13.158.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.158.84:33593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.158.126:32979,127.13.158.125:40219,127.13.158.124:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:49:47.202740 14993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 1
I20260808 11:49:47.203994 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.203527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36138)
0808 11:49:47.203731 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.203843 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 11:49:47.205112 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.204726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36150)
0808 11:49:47.204912 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.204987 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 11:49:47.205322 14993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
I20260808 11:49:47.273530 14993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 2
I20260808 11:49:47.274827 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.274246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36154)
0808 11:49:47.274488 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.274593 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 11:49:47.275823 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.275479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36164)
0808 11:49:47.275626 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.275703 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 11:49:47.276027 14993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
I20260808 11:49:47.296335 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } attempt: 5
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:49:47.309538 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.297148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.13.158.125:40219)
0808 11:49:47.297339 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.297344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.297365 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:49:47.297456 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.297461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.297528 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.297633 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.298474 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.299318 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.300628 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.301071 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.301217 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.304541 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.304545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.304548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.304606 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.307082 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.307794 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.307797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.307799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.307904 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.308313 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.308328 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.308330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.308589 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.308707 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.309132 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.309300 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 11:49:47.309629 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.297012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52186)
0808 11:49:47.297195 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.297216 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:47.297343 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:47.297680 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:47.297686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:47.298107 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:47.298305 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.298314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.299461 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.299474 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.300461 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.300475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.301226 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.301232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.301461 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.302116 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:47.302132 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:47.304280 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:47.307212 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.307218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.307222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.307675 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.308029 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.308034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.308037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.308210 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.308717 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:47.308722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:47.308888 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 11:49:47.309045 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 11:49:47.309295 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 11:49:47.310974 14878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:49:47.311982 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.311559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36170)
0808 11:49:47.311741 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.311833 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 11:49:47.313180 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.312673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36184)
0808 11:49:47.312857 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.312983 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 11:49:47.313434 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
I20260808 11:49:47.364673 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 1
I20260808 11:49:47.365983 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.365391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36192)
0808 11:49:47.365646 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.365765 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 11:49:47.367050 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.366641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36206)
0808 11:49:47.366838 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.366923 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 11:49:47.367202 14993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 3
W20260808 11:49:47.367352 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
I20260808 11:49:47.368171 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.367792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36208)
0808 11:49:47.367947 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.368035 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 11:49:47.369202 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.368786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36218)
0808 11:49:47.369017 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.369091 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260808 11:49:47.369438 14993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
I20260808 11:49:47.435611 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 2
I20260808 11:49:47.436884 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.436309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36222)
0808 11:49:47.436574 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.436682 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 11:49:47.437956 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.437546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36226)
0808 11:49:47.437717 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.437826 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 11:49:47.438228 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
I20260808 11:49:47.512677 14993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 4
I20260808 11:49:47.513980 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.513387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36236)
0808 11:49:47.513670 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.513777 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 11:49:47.515084 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.514622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36252)
0808 11:49:47.514861 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.514951 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 11:49:47.515300 14993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
W20260808 11:49:47.522897 15002 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:49:47.523255 15002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:49:47.523386 15002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:49:47.523463 15002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:49:47.530386 15002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:49:47.530462 15002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:49:47.530519 15002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:49:47.530568 15002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:49:47.530623 15002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:49:47.540688 15002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.158.84:33593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.158.126:32979,127.13.158.125:40219,127.13.158.124:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/krb5kdc/kudu_127.13.158.124.keytab
--principal=kudu/127.13.158.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.158.124:32897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.158.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:49:47.541891 15002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:49:47.543195 15002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 11:49:47.544464 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 3
I20260808 11:49:47.545671 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.545082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36262)
0808 11:49:47.545360 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.545466 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 11:49:47.546815 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.546406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36272)
0808 11:49:47.546577 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.546656 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 11:49:47.547130 14878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.158.124:32897: Network error: Client connection negotiation failed: client connection to 127.13.158.124:32897: connect: Connection refused (error 111)
W20260808 11:49:47.555673 15012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:47.555609 15014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:49:47.555720 15009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:49:47.556803 15002 server_base.cc:1034] running on GCE node
I20260808 11:49:47.557492 15002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:49:47.558816 15002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:49:47.560118 15002 hybrid_clock.cc:648] HybridClock initialized: now 1786189787560005 us; error 51 us; skew 500 ppm
I20260808 11:49:47.560497 15002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189787, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:49:47.563743 15002 init.cc:377] Logged in from keytab as kudu/127.13.158.124@KRBTEST.COM (short username kudu)
I20260808 11:49:47.565198 15002 webserver.cc:460] Webserver started at http://127.13.158.124:33145/ using document root <none> and password file <none>
I20260808 11:49:47.565876 15002 fs_manager.cc:359] Metadata directory not provided
I20260808 11:49:47.565992 15002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:49:47.566340 15002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:49:47.569196 15002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/data/instance:
uuid: "8fdd62751b6141dfa081add735572910"
format_stamp: "Formatted at 2026-08-08 11:49:47 on dist-test-slave-39l8"
server_key: "1c6af265a43158442b6084511ce0681b"
server_key_iv: "12237396c7522daa74d37ac2b92f7905"
server_key_version: "encryptionkey@0"
I20260808 11:49:47.569967 15002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/wal/instance:
uuid: "8fdd62751b6141dfa081add735572910"
format_stamp: "Formatted at 2026-08-08 11:49:47 on dist-test-slave-39l8"
server_key: "1c6af265a43158442b6084511ce0681b"
server_key_iv: "12237396c7522daa74d37ac2b92f7905"
server_key_version: "encryptionkey@0"
I20260808 11:49:47.575178 15002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260808 11:49:47.578513 15025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:49:47.579794 15002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:49:47.579962 15002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/data,/tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/wal
uuid: "8fdd62751b6141dfa081add735572910"
format_stamp: "Formatted at 2026-08-08 11:49:47 on dist-test-slave-39l8"
server_key: "1c6af265a43158442b6084511ce0681b"
server_key_iv: "12237396c7522daa74d37ac2b92f7905"
server_key_version: "encryptionkey@0"
I20260808 11:49:47.580192 15002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:49:47.603487 15002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:49:47.608111 15002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:49:47.608366 15002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:49:47.627338 15002 rpc_server.cc:307] RPC server started. Bound to: 127.13.158.124:32897
I20260808 11:49:47.627375 15104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.158.124:32897 every 8 connection(s)
I20260808 11:49:47.628643 15002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/data/info.pb
I20260808 11:49:47.632522 15105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:49:47.635358 13945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWrISxk/build/asan/bin/kudu as pid 15002
I20260808 11:49:47.635555 13945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWrISxk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189414476033-13945-0/minicluster-data/master-2/wal/instance
I20260808 11:49:47.636121 13945 external_mini_cluster.cc:1428] Setting key 3640d84f8e1b726e014aae7b36ca4231
I20260808 11:49:47.640483 15105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189787, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:49:47.658782 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.643352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.13.158.126:32979)
0808 11:49:47.643943 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.643952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.644423 (+ 471us) server_negotiation.cc:407] Connection header received
0808 11:49:47.644649 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.644654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.644725 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.644840 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.645960 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.646824 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.648135 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.648516 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.648645 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.652158 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.652166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.652172 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.652218 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.655678 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.656490 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.656494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.656495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.656644 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.656994 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.656999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.657002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.657323 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.657460 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.658127 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.658350 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":307,"threads_started":1}
I20260808 11:49:47.659623 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.643382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.126:32979 (local address 127.0.0.1:49080)
0808 11:49:47.644267 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.644310 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:47.644509 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:47.644922 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:47.644928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:47.645558 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:47.645808 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.645817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.646981 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.646986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.647988 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.647999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.648742 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.648750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.648897 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.649488 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:47.649508 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:47.651944 (+ 2436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:47.655807 (+ 3863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.655819 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.655833 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.656376 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.656758 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.656761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.656764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.656885 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.657449 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:47.657456 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:47.657711 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 11:49:47.657992 (+ 281us) client_negotiation.cc:240] Negotiation successful
0808 11:49:47.658634 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":54,"thread_start_us":520,"threads_started":1}
I20260808 11:49:47.659655 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.637977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.13.158.124:32897)
0808 11:49:47.638843 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.638854 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.638896 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:49:47.640528 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.640550 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.640960 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.641319 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.643094 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.645037 (+ 1943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.646359 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.647068 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.647275 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.651577 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.651586 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.651596 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.651628 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.655311 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.656055 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.656062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.656064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.656217 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.656640 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.656644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.656647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.657296 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.657601 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.657974 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.658427 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":608,"thread_start_us":379,"threads_started":1}
I20260808 11:49:47.661222 15105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189787, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.126@KRBTEST.COM
I20260808 11:49:47.676080 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.661936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.125:40219 (local address 127.0.0.1:52202)
0808 11:49:47.662334 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.662370 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:47.662475 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:47.663513 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:47.663518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:47.663969 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:47.664165 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.664180 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.665113 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.665119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.666256 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.666267 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.667266 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.667271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.667475 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.667993 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:47.668015 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:47.670364 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:47.673638 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.673643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.673648 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.674175 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.674569 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.674574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.674578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.674787 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.675294 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:47.675301 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:47.675475 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 11:49:47.675649 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 11:49:47.675833 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284}
I20260808 11:49:47.676607 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.662199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.13.158.125:40219)
0808 11:49:47.662416 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.662420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.662437 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:49:47.662615 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.662621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.662755 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.662858 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.664332 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.664968 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.666397 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.667103 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.667250 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.670553 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.670558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.670561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.670600 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.673493 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.674300 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.674304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.674306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.674430 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.674908 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.674911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.674913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.675158 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.675259 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.675762 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.675995 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:49:47.677523 15105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:49:47.678850 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.662763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.13.158.126:32979)
0808 11:49:47.663110 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.663115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.663130 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:49:47.663176 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.663179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.663240 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.663353 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.664780 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.665753 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.667292 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.667881 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.668028 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.672410 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.672415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.672418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.672447 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.675446 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.676301 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.676306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.676310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.676477 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.676945 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.676951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.676953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.677276 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.677538 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.677973 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.678186 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189787, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.124@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.158.125@KRBTEST.COM
I20260808 11:49:47.692266 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.678247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36290)
0808 11:49:47.678405 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.678425 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:47.678515 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:47.678855 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:47.678861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:47.679323 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:47.679463 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.679470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.680722 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.680730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.682185 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.682199 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.683130 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.683134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.683341 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.683845 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:47.683869 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:47.686503 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:47.689644 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.689650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.689653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.690170 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.690572 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.690575 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.690579 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.690699 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.691277 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:47.691281 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:47.691434 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 11:49:47.691576 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 11:49:47.691852 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 11:49:47.692296 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.678448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.13.158.124:32897)
0808 11:49:47.678592 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.678596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.678609 (+ 13us) server_negotiation.cc:407] Connection header received
0808 11:49:47.678644 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.678647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.678693 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.678816 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.679643 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.680581 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.682353 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.682993 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.683116 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.686694 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.686739 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.686747 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.686791 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.689507 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.690303 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.690312 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.690316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.690454 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.690884 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.690891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.690894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.691151 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.691293 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.691662 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.691976 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":56}
I20260808 11:49:47.695966 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.680520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52210 (local address 127.13.158.125:40219)
0808 11:49:47.680711 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.680716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.680738 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:49:47.680776 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.680779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.680829 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.680897 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.682448 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.683415 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.685226 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.685907 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.685999 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.690141 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.690146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.690149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.690178 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.693624 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.694192 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.694197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.694200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.694350 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.694684 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.694689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.694692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.695011 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.695292 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.695433 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.695642 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:49:47.697871 15105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: Bootstrap starting.
I20260808 11:49:47.700604 13945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:49:47.701721 15105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:47.702903 15105 log.cc:824] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:47.705672 15105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: No bootstrap required, opened a new log
I20260808 11:49:47.702872 14844 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0663431b6194438b856e868b4ac3a260"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49092
I20260808 11:49:47.712339 15105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } }
I20260808 11:49:47.712675 15105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fdd62751b6141dfa081add735572910, State: Initialized, Role: FOLLOWER
I20260808 11:49:47.713112 15105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } }
I20260808 11:49:47.714066 15116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.714517 15116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:47.715147 15105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:49:47.724184 15132 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 11:49:47.724334 14878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 4
W20260808 11:49:47.724684 15132 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.126@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
I20260808 11:49:47.722033 15002 master_runner.cc:186] Error getting master registration for 127.13.158.126:32979: OK, instance_id { permanent_uuid: "0663431b6194438b856e868b4ac3a260" instance_seqno: 1786189786579681 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:49:47.733510 15002 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 11:49:47.739101 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.724912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36296)
0808 11:49:47.725162 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.725182 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:47.725312 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:47.725811 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:47.725816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:47.726359 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:47.726569 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.726593 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.728131 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.728137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.728999 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.729011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.730070 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.730092 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.730312 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.730957 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:47.730983 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:47.733400 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:47.736670 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.736676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.736680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.737036 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.737420 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.737425 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.737428 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.737582 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.738144 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:47.738149 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:47.738289 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 11:49:47.738433 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 11:49:47.738699 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260808 11:49:47.739125 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.724978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.13.158.124:32897)
0808 11:49:47.725179 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.725189 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.725343 (+ 154us) server_negotiation.cc:407] Connection header received
0808 11:49:47.725460 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.725466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.725543 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.725636 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.726770 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.727979 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.729156 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.729938 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.730056 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.733600 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.733607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.733614 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.733659 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.736524 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.737169 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.737174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.737183 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.737307 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.737690 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.737696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.737699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.738023 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.738191 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.738532 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.738753 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 11:49:47.743429 14878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260: Bootstrap starting.
I20260808 11:49:47.746240 14878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:47.747247 14878 log.cc:824] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:47.749853 14878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260: No bootstrap required, opened a new log
I20260808 11:49:47.755055 14878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } }
I20260808 11:49:47.755405 14878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0663431b6194438b856e868b4ac3a260, State: Initialized, Role: FOLLOWER
I20260808 11:49:47.755949 14878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } }
I20260808 11:49:47.756963 15137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.757386 15137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:47.758109 14878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:47.758280 14844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 11:49:47.758497 14844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:47.762611 14844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } }
I20260808 11:49:47.764519 14844 leader_election.cc:288] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [CANDIDATE]: Term 1 election: Requested vote from peers 1b185c12f90342fea1067f0ee8fc1d3f (127.13.158.125:40219), 8fdd62751b6141dfa081add735572910 (127.13.158.124:32897)
I20260808 11:49:47.766238 15067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0663431b6194438b856e868b4ac3a260" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8fdd62751b6141dfa081add735572910"
I20260808 11:49:47.766772 15067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:47.770239 15067 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0663431b6194438b856e868b4ac3a260 in term 1.
I20260808 11:49:47.770923 14803 leader_election.cc:302] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0663431b6194438b856e868b4ac3a260, 8fdd62751b6141dfa081add735572910; no voters:
I20260808 11:49:47.771379 15137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:49:47.772086 15137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [term 1 LEADER]: Becoming Leader. State: Replica: 0663431b6194438b856e868b4ac3a260, State: Running, Role: LEADER
I20260808 11:49:47.772676 15137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } }
I20260808 11:49:47.790788 15148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0663431b6194438b856e868b4ac3a260. Latest consensus state: current_term: 1 leader_uuid: "0663431b6194438b856e868b4ac3a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.791261 15148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:47.797703 15163 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 11:49:47.802047 15164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:49:47.814502 14993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } attempt: 5
I20260808 11:49:47.801990 14774 master_runner.cc:186] Error getting master registration for 127.13.158.125:40219: OK, instance_id { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" instance_seqno: 1786189787092448 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:49:47.818415 14774 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 11:49:47.819340 14806 tablet.cc:2369] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:49:47.822009 15164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 11:49:47 dist-test-slave-39l8 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189787, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.158.125@KRBTEST.COM for kudu/127.13.158.124@KRBTEST.COM
I20260808 11:49:47.840303 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.815113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.158.124:32897 (local address 127.0.0.1:36302)
0808 11:49:47.815492 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 11:49:47.815511 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:49:47.815632 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:49:47.816473 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:49:47.816478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:49:47.816998 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:49:47.817221 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.817237 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.818741 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.818750 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.820255 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:49:47.820277 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.822031 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.822040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:49:47.822521 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.823568 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:49:47.823623 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:49:47.828140 (+ 4517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:49:47.834039 (+ 5899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.834047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.834074 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.834995 (+ 921us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.835738 (+ 743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:49:47.835742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:49:47.835745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:49:47.836033 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:49:47.837147 (+ 1114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:49:47.837151 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:49:47.837428 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 11:49:47.837812 (+ 384us) client_negotiation.cc:240] Negotiation successful
0808 11:49:47.838871 (+ 1059us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260808 11:49:47.840353 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.815843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.13.158.124:32897)
0808 11:49:47.816016 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.816022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.816040 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:49:47.816100 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.816106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.816198 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.816297 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.817408 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.818573 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.820520 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.821540 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.821707 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.828425 (+ 6718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.828434 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.828442 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.828489 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.833816 (+ 5327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.835180 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.835187 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.835192 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.835504 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.836240 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.836245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.836248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.836970 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.837866 (+ 896us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.838279 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.839190 (+ 911us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:49:47.841882 15067 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [term 1 FOLLOWER]: Refusing update from remote peer 0663431b6194438b856e868b4ac3a260: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:49:47.842854 15137 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:49:47.843464 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.784039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.13.158.126:32979)
0808 11:49:47.784317 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.784324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.797917 (+ 13593us) server_negotiation.cc:407] Connection header received
0808 11:49:47.798602 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.798607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.798692 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.798792 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.805014 (+ 6222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.806401 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.810009 (+ 3608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.810795 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.810893 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.831509 (+ 20616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.831518 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.831526 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.831587 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.835496 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.837128 (+ 1632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.837134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.837137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.837337 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.841023 (+ 3686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.841028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.841033 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.841725 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.841959 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.842643 (+ 684us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.842929 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260808 11:49:47.846624 14993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f: Bootstrap starting.
I20260808 11:49:47.851629 14993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f: Neither blocks nor log segments found. Creating new log.
I20260808 11:49:47.854189 14993 log.cc:824] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:49:47.859334 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.796555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.13.158.125:40219)
0808 11:49:47.796820 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.796830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.816073 (+ 19243us) server_negotiation.cc:407] Connection header received
0808 11:49:47.827367 (+ 11294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.827373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.827461 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.827534 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.837492 (+ 9958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.838593 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.846338 (+ 7745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.847030 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.847132 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.851070 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.851075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.851080 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.851114 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.855261 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.856257 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.856265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.856271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.856450 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.857019 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.857026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.857032 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.857477 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.857763 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.858483 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.858823 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 11:49:47.860867 15033 tablet.cc:2369] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:49:47.865806 14993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f: No bootstrap required, opened a new log
I20260808 11:49:47.871953 15116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0663431b6194438b856e868b4ac3a260. Latest consensus state: current_term: 1 leader_uuid: "0663431b6194438b856e868b4ac3a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.872349 15116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:47.877374 14993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } }
I20260808 11:49:47.877779 14993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b185c12f90342fea1067f0ee8fc1d3f, State: Initialized, Role: FOLLOWER
I20260808 11:49:47.878409 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:47.816662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.13.158.124:32897)
0808 11:49:47.827184 (+ 10522us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:47.827191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:47.830005 (+ 2814us) server_negotiation.cc:407] Connection header received
0808 11:49:47.830162 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:47.830167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:47.830240 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:47.830359 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:47.841593 (+ 11234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.842826 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.845696 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:47.846379 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:47.846495 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:47.857811 (+ 11316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:49:47.857822 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:49:47.857830 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:49:47.857884 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:49:47.862237 (+ 4353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.874874 (+ 12637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.874880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.874884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.875102 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:49:47.875652 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:49:47.875660 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:49:47.875670 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:49:47.876098 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:49:47.876364 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:47.876785 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 11:49:47.877291 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10384,"thread_start_us":10166,"threads_started":1}
I20260808 11:49:47.878322 14993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } }
I20260808 11:49:47.885041 15170 mvcc.cc:205] Tried to move back new op lower bound from 7316233370949685248 to 7316233370722062336. Current Snapshot: MvccSnapshot[applied={T|T < 7316233370949685248}]
I20260808 11:49:47.895352 15180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.895884 15180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:47.896659 14993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:49:47.896869 14948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0663431b6194438b856e868b4ac3a260" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1b185c12f90342fea1067f0ee8fc1d3f"
I20260808 11:49:47.897517 15171 mvcc.cc:205] Tried to move back new op lower bound from 7316233370949685248 to 7316233370722062336. Current Snapshot: MvccSnapshot[applied={T|T < 7316233370949685248}]
I20260808 11:49:47.897536 14947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:49:47.903295 14947 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [term 1 FOLLOWER]: Refusing update from remote peer 0663431b6194438b856e868b4ac3a260: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:49:47.908250 14948 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0663431b6194438b856e868b4ac3a260 in term 1.
I20260808 11:49:47.910612 15148 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:49:47.912328 15137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0663431b6194438b856e868b4ac3a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.912604 15137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:47.914021 15164 catalog_manager.cc:1347] Generated new cluster ID: 5d0df0286fac4fb1918e718791ea5d84
I20260808 11:49:47.914117 15164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:49:47.927410 15116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0663431b6194438b856e868b4ac3a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.927795 15116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:47.931666 15180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0663431b6194438b856e868b4ac3a260. Latest consensus state: current_term: 1 leader_uuid: "0663431b6194438b856e868b4ac3a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.931988 15180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:47.932068 15137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0663431b6194438b856e868b4ac3a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.932413 15137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260 [sys.catalog]: This master's current role is: LEADER
I20260808 11:49:47.934361 15188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0663431b6194438b856e868b4ac3a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0663431b6194438b856e868b4ac3a260" member_type: VOTER last_known_addr { host: "127.13.158.126" port: 32979 } } peers { permanent_uuid: "1b185c12f90342fea1067f0ee8fc1d3f" member_type: VOTER last_known_addr { host: "127.13.158.125" port: 40219 } } peers { permanent_uuid: "8fdd62751b6141dfa081add735572910" member_type: VOTER last_known_addr { host: "127.13.158.124" port: 32897 } } }
I20260808 11:49:47.934926 15188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:49:47.948634 15164 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:49:47.950443 15164 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:49:47.970827 15164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0663431b6194438b856e868b4ac3a260: Generated new TSK 0
I20260808 11:49:47.975682 15164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:49:47.998014 15207 catalog_manager.cc:1259] Loaded cluster ID: 5d0df0286fac4fb1918e718791ea5d84
I20260808 11:49:47.998241 15207 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f: loading cluster ID for follower catalog manager: success
I20260808 11:49:48.002203 15207 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f: acquiring CA information for follower catalog manager: success
I20260808 11:49:48.004750 15207 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b185c12f90342fea1067f0ee8fc1d3f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 11:49:48.146692 15111 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.135422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.13.158.126:32979)
0808 11:49:48.135664 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.135672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.143805 (+ 8133us) server_negotiation.cc:407] Connection header received
0808 11:49:48.145067 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.145091 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.145179 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.145257 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.146458 (+ 1201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49120: BlockingRecv error: recv got EOF from 127.0.0.1:49120 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 11:49:48.154384 14999 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.144342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52220 (local address 127.13.158.125:40219)
0808 11:49:48.144538 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.144544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.148191 (+ 3647us) server_negotiation.cc:407] Connection header received
0808 11:49:48.148355 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.148361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.148453 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.148595 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.154150 (+ 5555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52220: BlockingRecv error: recv got EOF from 127.0.0.1:52220 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 11:49:48.154495 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.151602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.13.158.124:32897)
0808 11:49:48.151778 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.151788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.152572 (+ 784us) server_negotiation.cc:407] Connection header received
0808 11:49:48.152834 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.152840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.152921 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.152989 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.154254 (+ 1265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36322: BlockingRecv error: recv got EOF from 127.0.0.1:36322 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:49:48.209414 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.193247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.13.158.125:40219)
0808 11:49:48.193427 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.193433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.200799 (+ 7366us) server_negotiation.cc:407] Connection header received
0808 11:49:48.201575 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.201582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.201805 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.201956 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.202731 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.203880 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.206030 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.206730 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.206840 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.207817 (+ 977us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.208561 (+ 744us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.208683 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.208929 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.209050 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 11:49:48.251344 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.239386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.13.158.124:32897)
0808 11:49:48.239586 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.239594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.249171 (+ 9577us) server_negotiation.cc:407] Connection header received
0808 11:49:48.249598 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.249605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.249747 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.249834 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.251062 (+ 1228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36332: BlockingRecv error: recv got EOF from 127.0.0.1:36332 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:49:48.300060 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.285104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.13.158.125:40219)
0808 11:49:48.285357 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.285364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.292759 (+ 7395us) server_negotiation.cc:407] Connection header received
0808 11:49:48.293258 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.293264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.293381 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.293451 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.294150 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.295406 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.297644 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.298333 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.298442 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.299147 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.299328 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.299396 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.299650 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.299742 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 11:49:48.343739 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.334001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.13.158.124:32897)
0808 11:49:48.334198 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.334204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.341687 (+ 7483us) server_negotiation.cc:407] Connection header received
0808 11:49:48.342227 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.342236 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.342357 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.342445 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.343485 (+ 1040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36338: BlockingRecv error: recv got EOF from 127.0.0.1:36338 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:49:48.392927 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.381405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52260 (local address 127.13.158.125:40219)
0808 11:49:48.381614 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.381621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.387229 (+ 5608us) server_negotiation.cc:407] Connection header received
0808 11:49:48.387514 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.387519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.387611 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.387728 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.388272 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.389336 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.391052 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.391521 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.391662 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.392115 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.392243 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.392345 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.392472 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.392565 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 11:49:48.401644 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.399411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36348 (local address 127.13.158.124:32897)
0808 11:49:48.399670 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.399679 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.400177 (+ 498us) server_negotiation.cc:407] Connection header received
0808 11:49:48.400394 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.400399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.400498 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.400607 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.401373 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36348: BlockingRecv error: recv got EOF from 127.0.0.1:36348 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:49:48.414332 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.408594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.13.158.125:40219)
0808 11:49:48.408877 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.408886 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.409365 (+ 479us) server_negotiation.cc:407] Connection header received
0808 11:49:48.409572 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.409577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.409676 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.409774 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.410180 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.411034 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.412381 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.413038 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.413186 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.413580 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.413693 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.413789 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.413933 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.414010 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
W20260808 11:49:48.422487 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.420897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.13.158.124:32897)
0808 11:49:48.421153 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.421159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.421183 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:49:48.421256 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.421261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.421346 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.421456 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.422282 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36354: BlockingRecv error: recv got EOF from 127.0.0.1:36354 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 11:49:48.439656 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.433971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52282 (local address 127.13.158.125:40219)
0808 11:49:48.434206 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.434215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.434789 (+ 574us) server_negotiation.cc:407] Connection header received
0808 11:49:48.434999 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.435004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.435091 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.435199 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.435783 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.436506 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.437870 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.438327 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.438469 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.438887 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.438990 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.439101 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.439219 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.439298 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 11:49:48.447665 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.445427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36370 (local address 127.13.158.124:32897)
0808 11:49:48.445668 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.445676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.446227 (+ 551us) server_negotiation.cc:407] Connection header received
0808 11:49:48.446435 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.446441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.446525 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.446636 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.447417 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36370: BlockingRecv error: recv got EOF from 127.0.0.1:36370 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:49:48.472765 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.466648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.13.158.125:40219)
0808 11:49:48.466904 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.466914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.467437 (+ 523us) server_negotiation.cc:407] Connection header received
0808 11:49:48.467740 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.467750 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.467817 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.467921 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.468345 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.469356 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.470804 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.471352 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.471507 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.471941 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.472079 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.472184 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.472351 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.472436 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
W20260808 11:49:48.481190 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.478726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36372 (local address 127.13.158.124:32897)
0808 11:49:48.478985 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.478995 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.479544 (+ 549us) server_negotiation.cc:407] Connection header received
0808 11:49:48.479752 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.479758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.479853 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.479937 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.480969 (+ 1032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36372: BlockingRecv error: recv got EOF from 127.0.0.1:36372 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 11:49:48.522794 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.516831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.13.158.125:40219)
0808 11:49:48.517113 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.517122 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.517677 (+ 555us) server_negotiation.cc:407] Connection header received
0808 11:49:48.517921 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.517926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.518022 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.518149 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.518626 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.519456 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.520839 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.521356 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.521494 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.521919 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.522020 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.522122 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.522295 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.522376 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 11:49:48.532757 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.529116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36382 (local address 127.13.158.124:32897)
0808 11:49:48.529354 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.529363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.531248 (+ 1885us) server_negotiation.cc:407] Connection header received
0808 11:49:48.531457 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.531467 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.531576 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.531687 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.532518 (+ 831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36382: BlockingRecv error: recv got EOF from 127.0.0.1:36382 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 11:49:48.605844 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.600490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.13.158.125:40219)
0808 11:49:48.600763 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.600771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.601076 (+ 305us) server_negotiation.cc:407] Connection header received
0808 11:49:48.601333 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.601337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.601419 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.601534 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.602021 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.602843 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.604178 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.604617 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.604760 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.605159 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.605264 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.605362 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.605507 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.605583 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 11:49:48.612896 15167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:49:48.611212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.13.158.124:32897)
0808 11:49:48.611451 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.611459 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.611646 (+ 187us) server_negotiation.cc:407] Connection header received
0808 11:49:48.611787 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.611791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.611860 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.611940 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:49:48.612651 (+ 711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36390: BlockingRecv error: recv got EOF from 127.0.0.1:36390 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 11:49:48.726676 15132 catalog_manager.cc:1259] Loaded cluster ID: 5d0df0286fac4fb1918e718791ea5d84
I20260808 11:49:48.726882 15132 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: loading cluster ID for follower catalog manager: success
I20260808 11:49:48.729374 15132 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: acquiring CA information for follower catalog manager: success
I20260808 11:49:48.731341 15132 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fdd62751b6141dfa081add735572910: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:49:48.754189 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.748462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.13.158.125:40219)
0808 11:49:48.748711 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.748719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.749046 (+ 327us) server_negotiation.cc:407] Connection header received
0808 11:49:48.749293 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.749299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.749394 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.749494 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.749912 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.750635 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.752215 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.752785 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.752933 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.753441 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.753573 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.753659 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.753793 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.753859 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 11:49:48.765115 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.759781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.13.158.124:32897)
0808 11:49:48.760012 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.760018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.760220 (+ 202us) server_negotiation.cc:407] Connection header received
0808 11:49:48.760386 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.760390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.760543 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.760652 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.761054 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.761848 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.763069 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.763471 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.763561 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.764043 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.764556 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.764679 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.764723 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.764786 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 11:49:48.778590 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.771595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.13.158.125:40219)
0808 11:49:48.771782 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.771787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.772110 (+ 323us) server_negotiation.cc:407] Connection header received
0808 11:49:48.772440 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.772449 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.772566 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.772637 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.773471 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.774431 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.776645 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.777261 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.777401 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.777877 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.778016 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.778108 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.778287 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.778366 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:49:48.778554 15380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.770800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.13.158.126:32979)
0808 11:49:48.771335 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.771342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.771428 (+ 86us) server_negotiation.cc:407] Connection header received
0808 11:49:48.771725 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.771731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.771935 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.772178 (+ 243us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.772913 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.774043 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.775850 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.776292 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.776435 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.777064 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.777567 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.777696 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.778155 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.778253 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":276,"threads_started":1}
I20260808 11:49:48.779011 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:49:48.772243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.13.158.124:32897)
0808 11:49:48.772402 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 11:49:48.772407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:49:48.773032 (+ 625us) server_negotiation.cc:407] Connection header received
0808 11:49:48.773259 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:49:48.773264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:49:48.773350 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:49:48.773443 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:49:48.773937 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.774952 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.776969 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:49:48.777465 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:49:48.777592 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:49:48.778060 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:49:48.778214 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:49:48.778305 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 11:49:48.778481 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 11:49:48.778556 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:49:48.783954 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14774
I20260808 11:49:48.784922 14870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:48.895545 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14774
W20260808 11:49:48.921989 14909 connection.cc:535] server connection from 127.0.0.1:52186 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 11:49:48.922459 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 14885
I20260808 11:49:48.923651 14983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:49.032783 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 14885
I20260808 11:49:49.059526 13945 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWrISxk/build/asan/bin/kudu pid 15002
I20260808 11:49:49.060725 15095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:49:49.169847 13945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWrISxk/build/asan/bin/kudu with pid 15002
2026-08-08T11:49:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4752 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9957 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (374665 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST